[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:42 dist-test-slave-g170 krb5kdc[17628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:42 dist-test-slave-g170 krb5kdc[17628](info): set up 2 sockets
Aug 07 02:26:42 dist-test-slave-g170 krb5kdc[17628](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 02:26:43.564695 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:26:43 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:26:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:26:43.696410 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41263
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:43.816726 17647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:43.817131 17647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:43.817229 17647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:43.821656 17647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:26:43.821768 17647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:43.821811 17647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:43.821877 17647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:26:43.821920 17647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:26:43.827654 17647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:43.828912 17647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:43.830082 17647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:43.839618 17655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:43.839623 17661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:43.839785 17654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:43.839907 17647 server_base.cc:1034] running on GCE node
I20260807 02:26:43.840612 17647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:43.841939 17647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:43.843144 17647 hybrid_clock.cc:648] HybridClock initialized: now 1786069603843092 us; error 68 us; skew 500 ppm
Aug 07 02:26:43 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069603, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:43.852398 17647 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:26:43.853513 17647 webserver.cc:460] Webserver started at http://127.17.53.62:42297/ using document root <none> and password file <none>
I20260807 02:26:43.854090 17647 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:43.854163 17647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:43.854350 17647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:43.856460 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "1da9455a3c594b74bfa5dd2f9da8620d"
format_stamp: "Formatted at 2026-08-07 02:26:43 on dist-test-slave-g170"
server_key: "41bc4853e0313ce60fc1af1bc73ada25"
server_key_iv: "dbc089e83dc43dfa09ee5c8cccb598fe"
server_key_version: "encryptionkey@0"
I20260807 02:26:43.857126 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "1da9455a3c594b74bfa5dd2f9da8620d"
format_stamp: "Formatted at 2026-08-07 02:26:43 on dist-test-slave-g170"
server_key: "41bc4853e0313ce60fc1af1bc73ada25"
server_key_iv: "dbc089e83dc43dfa09ee5c8cccb598fe"
server_key_version: "encryptionkey@0"
I20260807 02:26:43.861258 17647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:26:43.864269 17674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:43.865868 17647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:43.866037 17647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "1da9455a3c594b74bfa5dd2f9da8620d"
format_stamp: "Formatted at 2026-08-07 02:26:43 on dist-test-slave-g170"
server_key: "41bc4853e0313ce60fc1af1bc73ada25"
server_key_iv: "dbc089e83dc43dfa09ee5c8cccb598fe"
server_key_version: "encryptionkey@0"
I20260807 02:26:43.866165 17647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:43.884060 17647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:43.887861 17647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:43.888092 17647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:43.896497 17647 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:41263
I20260807 02:26:43.896521 17749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:41263 every 8 connection(s)
I20260807 02:26:43.897626 17647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:26:43.901650 17750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:43.904456 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17647
I20260807 02:26:43.904647 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:26:43.906520 17620 external_mini_cluster.cc:1428] Setting key 6b966279ca1b16cc25eb8531ed10f00f
I20260807 02:26:43.909600 17750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d: Bootstrap starting.
I20260807 02:26:43.913033 17750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:43.914149 17750 log.cc:824] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:43.916687 17750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d: No bootstrap required, opened a new log
I20260807 02:26:43.921998 17750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } }
I20260807 02:26:43.922271 17750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:43.922331 17750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da9455a3c594b74bfa5dd2f9da8620d, State: Initialized, Role: FOLLOWER
I20260807 02:26:43.922889 17750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } }
I20260807 02:26:43.923094 17750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:26:43.923172 17750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:26:43.923278 17750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:43.924474 17750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } }
I20260807 02:26:43.924909 17750 leader_election.cc:302] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1da9455a3c594b74bfa5dd2f9da8620d; no voters:
I20260807 02:26:43.925289 17750 leader_election.cc:288] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:26:43.925364 17755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:43.925595 17755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [term 1 LEADER]: Becoming Leader. State: Replica: 1da9455a3c594b74bfa5dd2f9da8620d, State: Running, Role: LEADER
I20260807 02:26:43.925933 17755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } }
I20260807 02:26:43.926716 17750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:26:43.927879 17756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } } }
I20260807 02:26:43.928011 17756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:43.928323 17759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1da9455a3c594b74bfa5dd2f9da8620d. Latest consensus state: current_term: 1 leader_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1da9455a3c594b74bfa5dd2f9da8620d" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41263 } } }
I20260807 02:26:43.928431 17759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:43.929014 17772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 02:26:43 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:43.933213 17772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:26:43.937350 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:43.910868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.17.53.62:41263)
0807 02:26:43.911421 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:43.911440 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:43.912566 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 02:26:43.917154 (+ 4588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:43.917172 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:43.917515 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:43.917858 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:43.921154 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:43.922111 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:43.924071 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:43.924579 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:43.924832 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:43.930310 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:43.930344 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:43.930364 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:43.930423 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:43.932914 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:43.933601 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:43.933607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:43.933612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:43.933768 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:43.934264 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:43.934267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:43.934269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:43.934796 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:43.935020 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:43.935858 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 02:26:43.936131 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":175,"threads_started":1}
I20260807 02:26:43.939867 17772 catalog_manager.cc:1347] Generated new cluster ID: 6ffc2ee67fd9427bbc203075777be03c
I20260807 02:26:43.939931 17772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:26:43.956244 17772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:26:43.957346 17772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:26:43.964430 17772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1da9455a3c594b74bfa5dd2f9da8620d: Generated new TSK 0
I20260807 02:26:43.965261 17772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:26:44.000660 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:44.124976 17792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:44.125226 17792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:44.125298 17792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:44.129549 17792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:44.129606 17792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:44.129694 17792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:26:44.134892 17792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:44.135980 17792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:44.137097 17792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:44.147111 17802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.147113 17799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.147137 17798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:44.147276 17792 server_base.cc:1034] running on GCE node
I20260807 02:26:44.147918 17792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:44.148577 17792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:44.149782 17792 hybrid_clock.cc:648] HybridClock initialized: now 1786069604149748 us; error 45 us; skew 500 ppm
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:44.152725 17792 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:26:44.153628 17792 webserver.cc:460] Webserver started at http://127.17.53.1:34997/ using document root <none> and password file <none>
I20260807 02:26:44.154182 17792 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:44.154242 17792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:44.154408 17792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:44.156540 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "7fc089e7dba842aab3e1b245d6c22266"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "9949b892ac15fdfd2578001cb3b68780"
server_key_iv: "33fc0cc4511596061a4716e3ad4db0c7"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.157166 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "7fc089e7dba842aab3e1b245d6c22266"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "9949b892ac15fdfd2578001cb3b68780"
server_key_iv: "33fc0cc4511596061a4716e3ad4db0c7"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.161015 17792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:26:44.163542 17810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.164983 17792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:44.165088 17792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "7fc089e7dba842aab3e1b245d6c22266"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "9949b892ac15fdfd2578001cb3b68780"
server_key_iv: "33fc0cc4511596061a4716e3ad4db0c7"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.165235 17792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:44.192276 17792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:44.196380 17792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:44.196693 17792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:44.197561 17792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:44.198607 17792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:44.198689 17792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.198760 17792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:44.198812 17792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.210419 17792 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:43533
I20260807 02:26:44.210477 17994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:43533 every 8 connection(s)
I20260807 02:26:44.211673 17792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:26:44.219310 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17792
I20260807 02:26:44.219450 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:26:44.219758 17620 external_mini_cluster.cc:1428] Setting key b36392b8863fd7d70f522a36999cadaa
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:44.229180 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.214455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35307 (local address 127.17.53.62:41263)
0807 02:26:44.214642 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:44.214646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:44.215669 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 02:26:44.217104 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:44.217107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:44.217160 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:44.217241 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:44.218583 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.219315 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.220886 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.221227 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.221354 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.224375 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:44.224379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:44.224389 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:44.224417 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:44.226168 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.226998 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.227001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.227002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.227093 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.227623 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.227626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.227627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.227914 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:44.228143 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:44.228822 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 02:26:44.229000 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 02:26:44.230166 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.214881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41263 (local address 127.17.53.1:35307)
0807 02:26:44.215488 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:44.215536 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:44.216872 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:44.217450 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:44.217457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:44.217848 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:44.218408 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.218423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.219521 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.219526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.220732 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.220744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.221440 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.221445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.221639 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.222356 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:44.222374 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:44.224143 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:44.226371 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.226388 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.226418 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.226884 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.227213 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.227217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.227219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.227494 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.228136 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:44.228144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:44.228487 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 02:26:44.228718 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 02:26:44.229186 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":145,"threads_started":1}
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:26:44.231768 17995 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41263
I20260807 02:26:44.232079 17995 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:44.232717 17995 heartbeater.cc:511] Master 127.17.53.62:41263 requested a full tablet report, sending...
I20260807 02:26:44.234592 17694 ts_manager.cc:194] Registered new tserver with Master: 7fc089e7dba842aab3e1b245d6c22266 (127.17.53.1:43533)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:26:44.236267 17694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:35307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:26:44.255618 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:44.381584 18001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:44.381886 18001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:44.381963 18001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:44.386071 18001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:44.386157 18001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:44.386247 18001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:26:44.391463 18001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:44.392527 18001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:44.393635 18001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:44.403666 18011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.403678 18007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.403777 18009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:44.403831 18001 server_base.cc:1034] running on GCE node
I20260807 02:26:44.404381 18001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:44.405092 18001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:44.406313 18001 hybrid_clock.cc:648] HybridClock initialized: now 1786069604406297 us; error 26 us; skew 500 ppm
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:44.409320 18001 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:26:44.410269 18001 webserver.cc:460] Webserver started at http://127.17.53.2:34275/ using document root <none> and password file <none>
I20260807 02:26:44.410849 18001 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:44.410945 18001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:44.411229 18001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:44.413440 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "3464491e653e46c1b82ca2d959f05bb8"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "5e5ef982c042959aa20d9b17e45f9298"
server_key_iv: "2a109df7d360c8ec82b4acbfbb4feaa1"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.414043 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "3464491e653e46c1b82ca2d959f05bb8"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "5e5ef982c042959aa20d9b17e45f9298"
server_key_iv: "2a109df7d360c8ec82b4acbfbb4feaa1"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.418165 18001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:26:44.420866 18020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.422340 18001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:26:44.422508 18001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "3464491e653e46c1b82ca2d959f05bb8"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "5e5ef982c042959aa20d9b17e45f9298"
server_key_iv: "2a109df7d360c8ec82b4acbfbb4feaa1"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.422645 18001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:44.454034 18001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:44.457541 18001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:44.457839 18001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:44.458580 18001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:44.459543 18001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:44.459623 18001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.459753 18001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:44.459867 18001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.471632 18001 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:43407
I20260807 02:26:44.471654 18194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:43407 every 8 connection(s)
I20260807 02:26:44.472615 18001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:26:44.473701 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18001
I20260807 02:26:44.473821 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:26:44.474067 17620 external_mini_cluster.cc:1428] Setting key 7474d3a8ea68bfb08827b13dce75b8b2
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:26:44.489861 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.474845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:36437 (local address 127.17.53.62:41263)
0807 02:26:44.475007 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:44.475011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:44.476198 (+ 1187us) server_negotiation.cc:407] Connection header received
0807 02:26:44.478280 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:44.478285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:44.478348 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:44.478466 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:44.479742 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.480600 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.482010 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.482247 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.482372 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.485420 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:44.485424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:44.485426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:44.485455 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:44.487025 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.487716 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.487719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.487721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.487845 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.488294 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.488299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.488301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.488554 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:44.488681 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:44.489481 (+ 800us) server_negotiation.cc:299] Negotiation successful
0807 02:26:44.489629 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 02:26:44.491127 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.475367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41263 (local address 127.17.53.2:36437)
0807 02:26:44.476000 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:44.476061 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:44.477970 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:44.478630 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:44.478637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:44.479024 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:44.479583 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.479598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.480816 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.480821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.481869 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.481880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.482423 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.482427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.482606 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.483311 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:44.483331 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:44.485216 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:44.487172 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.487178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.487198 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.487511 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.487948 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.487956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.487958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.488126 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.488679 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:44.488687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:44.489125 (+ 438us) client_negotiation.cc:769] Sending connection context
0807 02:26:44.489375 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 02:26:44.489833 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":166,"threads_started":1}
I20260807 02:26:44.492605 18195 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41263
I20260807 02:26:44.492935 18195 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:44.493860 18195 heartbeater.cc:511] Master 127.17.53.62:41263 requested a full tablet report, sending...
I20260807 02:26:44.495353 17694 ts_manager.cc:194] Registered new tserver with Master: 3464491e653e46c1b82ca2d959f05bb8 (127.17.53.2:43407)
I20260807 02:26:44.496025 17694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:36437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:26:44.508916 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:44.625859 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:44.626195 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:44.626305 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:44.630728 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:44.630795 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:44.630858 18200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:26:44.636157 18200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:44.637392 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:44.638447 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:44.648343 18211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.648355 18208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:44.648360 18207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:44.648729 18200 server_base.cc:1034] running on GCE node
I20260807 02:26:44.649216 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:44.649878 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:44.651053 18200 hybrid_clock.cc:648] HybridClock initialized: now 1786069604651030 us; error 34 us; skew 500 ppm
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:44.653946 18200 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:26:44.654809 18200 webserver.cc:460] Webserver started at http://127.17.53.3:41505/ using document root <none> and password file <none>
I20260807 02:26:44.655349 18200 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:44.655409 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:44.655575 18200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:44.657716 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "d31cc2235c5440e38d2fc55382821f21"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "86e0a17bf26e852a5a38e57b1e816ff3"
server_key_iv: "1baef5cabea98e2f0a6b245e8c865bb6"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.658250 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "d31cc2235c5440e38d2fc55382821f21"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "86e0a17bf26e852a5a38e57b1e816ff3"
server_key_iv: "1baef5cabea98e2f0a6b245e8c865bb6"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.662009 18200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 02:26:44.664505 18223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.665961 18200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:44.666113 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "d31cc2235c5440e38d2fc55382821f21"
format_stamp: "Formatted at 2026-08-07 02:26:44 on dist-test-slave-g170"
server_key: "86e0a17bf26e852a5a38e57b1e816ff3"
server_key_iv: "1baef5cabea98e2f0a6b245e8c865bb6"
server_key_version: "encryptionkey@0"
I20260807 02:26:44.666288 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:44.687037 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:44.690528 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:44.690757 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:44.691500 18200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:44.692449 18200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:44.692528 18200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.692602 18200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:44.692654 18200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:44.703622 18200 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:35885
I20260807 02:26:44.703703 18403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:35885 every 8 connection(s)
I20260807 02:26:44.704646 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:26:44.706969 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18200
I20260807 02:26:44.707108 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:26:44.707383 17620 external_mini_cluster.cc:1428] Setting key acca8b51d844af007012cf5134ab45d9
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:44.719954 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.706798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:56839 (local address 127.17.53.62:41263)
0807 02:26:44.706991 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:44.706997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:44.707995 (+ 998us) server_negotiation.cc:407] Connection header received
0807 02:26:44.709601 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:44.709609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:44.709665 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:44.709763 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:44.711011 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.711567 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.713050 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.713324 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.713420 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.716026 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:44.716030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:44.716032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:44.716058 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:44.717623 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.718188 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.718192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.718194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.718280 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.718695 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.718698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.718699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.718953 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:44.719054 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:44.719686 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 02:26:44.719809 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 02:26:44.720983 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.707273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41263 (local address 127.17.53.3:56839)
0807 02:26:44.707815 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:44.707872 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:44.709393 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:44.709953 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:44.709962 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:44.710357 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:44.710872 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.710887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.711723 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.711727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.712874 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:44.712887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.713492 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.713495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:44.713666 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.714261 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:44.714279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:44.715837 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:44.717772 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.717778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.717793 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.718075 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.718403 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:44.718406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:44.718407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:44.718562 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:44.719073 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:44.719079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:44.719380 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 02:26:44.719569 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 02:26:44.719859 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":175,"threads_started":1}
I20260807 02:26:44.722563 18404 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41263
I20260807 02:26:44.722935 18404 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:44.723598 18404 heartbeater.cc:511] Master 127.17.53.62:41263 requested a full tablet report, sending...
I20260807 02:26:44.724829 17694 ts_manager.cc:194] Registered new tserver with Master: d31cc2235c5440e38d2fc55382821f21 (127.17.53.3:35885)
I20260807 02:26:44.725432 17694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:56839
I20260807 02:26:44.735489 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:44.763530 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.753965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.17.53.1:43533)
0807 02:26:44.754346 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:44.754353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:44.754378 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:26:44.754431 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:44.754436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:44.754627 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:44.754751 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:44.755487 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.756239 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.757206 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.757602 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.757706 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.760146 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:44.760156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:44.760173 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:44.760199 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:44.761903 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.762315 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.762321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.762322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.762406 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.762645 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.762647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.762648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.762927 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:44.763103 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:44.763263 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 02:26:44.763391 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":158,"threads_started":1}
W20260807 02:26:44.764544 17864 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53086
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[17628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:44.792516 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:44.783892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53092 (local address 127.17.53.1:43533)
0807 02:26:44.784068 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:44.784072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:44.784135 (+ 63us) server_negotiation.cc:407] Connection header received
0807 02:26:44.784243 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:44.784245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:44.784301 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:44.784371 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:44.785077 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.785571 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.786714 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:44.787031 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:44.787114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:44.789485 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:44.789489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:44.789491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:44.789513 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:44.790970 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.791403 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.791406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.791407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.791490 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:44.791781 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:44.791783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:44.791784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:44.791954 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:44.792057 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:44.792231 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 02:26:44.792328 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 02:26:44.793800 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17792
I20260807 02:26:44.806428 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18001
I20260807 02:26:44.818279 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18200
I20260807 02:26:44.842288 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17647
2026-08-07T02:26:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2598 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[18428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[18428](info): set up 2 sockets
Aug 07 02:26:44 dist-test-slave-g170 krb5kdc[18428](info): commencing operation
krb5kdc: starting...
W20260807 02:26:46.140244 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:26:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:26:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:26:46.280158 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37847
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:46.395375 18444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:46.395656 18444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:46.395697 18444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:46.399781 18444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:26:46.399860 18444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:46.399886 18444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:46.399904 18444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:26:46.399924 18444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:26:46.405439 18444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:46.406635 18444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:46.407701 18444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:46.416564 18454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.416564 18455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.416556 18458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:46.416867 18444 server_base.cc:1034] running on GCE node
I20260807 02:26:46.417502 18444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:46.418607 18444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:46.419785 18444 hybrid_clock.cc:648] HybridClock initialized: now 1786069606419764 us; error 30 us; skew 500 ppm
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:46.422659 18444 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:26:46.423573 18444 webserver.cc:460] Webserver started at http://127.17.53.62:45687/ using document root <none> and password file <none>
I20260807 02:26:46.424095 18444 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:46.424154 18444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:46.424330 18444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:46.426424 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "58d7014e574a4ff8b9cc4319e85a23fe"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "5bc389552322fea6ef117bac92c8af7e"
server_key_iv: "30dd5f295b21e79b9b7a9ca5f4445d08"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.426975 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "58d7014e574a4ff8b9cc4319e85a23fe"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "5bc389552322fea6ef117bac92c8af7e"
server_key_iv: "30dd5f295b21e79b9b7a9ca5f4445d08"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.430922 18444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:26:46.433718 18467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.435098 18444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:46.435271 18444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "58d7014e574a4ff8b9cc4319e85a23fe"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "5bc389552322fea6ef117bac92c8af7e"
server_key_iv: "30dd5f295b21e79b9b7a9ca5f4445d08"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.435397 18444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:46.457003 18444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:46.459995 18444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:46.460208 18444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:46.468099 18444 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:37847
I20260807 02:26:46.468132 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:37847 every 8 connection(s)
I20260807 02:26:46.469146 18444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:26:46.471951 18553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:46.477527 18553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: Bootstrap starting.
I20260807 02:26:46.477900 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18444
I20260807 02:26:46.477996 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:26:46.478273 17620 external_mini_cluster.cc:1428] Setting key 71e9a37f0908d48cc53b5186b8e28554
I20260807 02:26:46.480242 18553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:46.481194 18553 log.cc:824] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:46.483395 18553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: No bootstrap required, opened a new log
I20260807 02:26:46.487437 18553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } }
I20260807 02:26:46.487635 18553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:46.487669 18553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d7014e574a4ff8b9cc4319e85a23fe, State: Initialized, Role: FOLLOWER
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:46.488181 18553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } }
I20260807 02:26:46.488308 18553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:26:46.488394 18553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:26:46.488498 18553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:46.489670 18553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } }
I20260807 02:26:46.490029 18553 leader_election.cc:302] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58d7014e574a4ff8b9cc4319e85a23fe; no voters:
I20260807 02:26:46.490427 18553 leader_election.cc:288] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:26:46.490500 18559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:46.490715 18559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [term 1 LEADER]: Becoming Leader. State: Replica: 58d7014e574a4ff8b9cc4319e85a23fe, State: Running, Role: LEADER
I20260807 02:26:46.491012 18559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } }
I20260807 02:26:46.491742 18553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:26:46.492192 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } } }
I20260807 02:26:46.492324 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:46.492652 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58d7014e574a4ff8b9cc4319e85a23fe. Latest consensus state: current_term: 1 leader_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d7014e574a4ff8b9cc4319e85a23fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37847 } } }
I20260807 02:26:46.492784 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:46.494510 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:46.479641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.17.53.62:37847)
0807 02:26:46.480052 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:46.480074 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:46.480120 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:26:46.481010 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:46.481036 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:46.481464 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:46.481798 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:46.482595 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.484005 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.484909 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.485618 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.485779 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:46.488547 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:46.488552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:46.488564 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:46.488590 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:46.490666 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:46.491254 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:46.491259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:46.491264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:46.491401 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:46.491700 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:46.491704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:46.491706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:46.492235 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:46.493115 (+ 880us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:46.493264 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 02:26:46.493557 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
W20260807 02:26:46.496865 18584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:26:46.496946 18584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:26:46.497017 18571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:26:46.497736 18571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:26:46.503669 18571 catalog_manager.cc:1347] Generated new cluster ID: 45eefa6dc86249c6b0626f773298df98
I20260807 02:26:46.503767 18571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:26:46.514914 18571 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:26:46.515887 18571 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:26:46.521337 18571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58d7014e574a4ff8b9cc4319e85a23fe: Generated new TSK 0
I20260807 02:26:46.522104 18571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:26:46.559927 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:46.690310 18595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:46.690557 18595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:46.690599 18595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:46.694721 18595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:46.694779 18595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:46.694865 18595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:26:46.700035 18595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:46.701108 18595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:46.702133 18595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:46.712220 18604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.712185 18601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.712198 18600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:46.712499 18595 server_base.cc:1034] running on GCE node
I20260807 02:26:46.713080 18595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:46.713743 18595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:46.714905 18595 hybrid_clock.cc:648] HybridClock initialized: now 1786069606714873 us; error 43 us; skew 500 ppm
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:46.717772 18595 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:26:46.718647 18595 webserver.cc:460] Webserver started at http://127.17.53.1:45173/ using document root <none> and password file <none>
I20260807 02:26:46.719282 18595 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:46.719352 18595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:46.719592 18595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:46.721781 18595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "b60e68ffc2824cc2823b50a6b1709d5f"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "6ab8dc39bffe451ca2da184618595985"
server_key_iv: "4df296639875c405b03a793ce5c2e6d0"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.722432 18595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "b60e68ffc2824cc2823b50a6b1709d5f"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "6ab8dc39bffe451ca2da184618595985"
server_key_iv: "4df296639875c405b03a793ce5c2e6d0"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.727145 18595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:26:46.730193 18613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.731619 18595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:26:46.731789 18595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "b60e68ffc2824cc2823b50a6b1709d5f"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "6ab8dc39bffe451ca2da184618595985"
server_key_iv: "4df296639875c405b03a793ce5c2e6d0"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.731914 18595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:46.751881 18595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:46.755180 18595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:46.755402 18595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:46.756052 18595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:46.756953 18595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:46.757030 18595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.757105 18595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:46.757149 18595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.766669 18595 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:36507
I20260807 02:26:46.766695 18790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:36507 every 8 connection(s)
I20260807 02:26:46.767554 18595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:26:46.768496 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18595
I20260807 02:26:46.768594 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:26:46.768941 17620 external_mini_cluster.cc:1428] Setting key 4092f61395d46f3688f0326c327373af
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:26:46.783056 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:46.769663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:51133 (local address 127.17.53.62:37847)
0807 02:26:46.769820 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:46.769826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:46.770866 (+ 1040us) server_negotiation.cc:407] Connection header received
0807 02:26:46.772329 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:46.772334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:46.772408 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:46.772514 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:46.773775 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.774610 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.775795 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.776287 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.776406 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:46.779241 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:46.779245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:46.779249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:46.779288 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:46.781032 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:46.781522 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:46.781524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:46.781525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:46.781602 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:46.782013 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:46.782015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:46.782017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:46.782198 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:46.782276 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:46.782827 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 02:26:46.782931 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:26:46.784206 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:46.770157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37847 (local address 127.17.53.1:51133)
0807 02:26:46.770697 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:46.770749 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:46.772051 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:46.772684 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:46.772690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:46.773106 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:46.773606 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:46.773621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.774763 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.774767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:46.775650 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:46.775661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:46.776449 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:46.776451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:46.776593 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:46.777326 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:46.777342 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:46.779069 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:46.781161 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:46.781166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:46.781177 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:46.781428 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:46.781695 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:46.781721 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:46.781726 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:46.781919 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:46.782295 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:46.782300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:46.782533 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 02:26:46.782808 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 02:26:46.783172 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":176,"threads_started":1}
I20260807 02:26:46.785696 18791 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37847
I20260807 02:26:46.785964 18791 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:46.786494 18791 heartbeater.cc:511] Master 127.17.53.62:37847 requested a full tablet report, sending...
I20260807 02:26:46.788360 18496 ts_manager.cc:194] Registered new tserver with Master: b60e68ffc2824cc2823b50a6b1709d5f (127.17.53.1:36507)
I20260807 02:26:46.790112 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:51133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:26:46.800450 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:46.917948 18799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:46.918206 18799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:46.918267 18799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:46.922467 18799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:46.922529 18799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:46.922588 18799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:26:46.927811 18799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:46.928983 18799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:46.930033 18799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:46.940058 18806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.940050 18809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:46.940043 18811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:46.940368 18799 server_base.cc:1034] running on GCE node
I20260807 02:26:46.941020 18799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:46.941747 18799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:46.942976 18799 hybrid_clock.cc:648] HybridClock initialized: now 1786069606942957 us; error 33 us; skew 500 ppm
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:46.945961 18799 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:26:46.946861 18799 webserver.cc:460] Webserver started at http://127.17.53.2:36659/ using document root <none> and password file <none>
I20260807 02:26:46.947453 18799 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:46.947547 18799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:46.947793 18799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:46.949934 18799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "a105df100caa4f4e98b06a53ad4266ae"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "fb9ff0d9ecb6a6bf506e0273ebf7a028"
server_key_iv: "3e0baa71b6eab778a4d6b3e5d5014b6a"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.950520 18799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "a105df100caa4f4e98b06a53ad4266ae"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "fb9ff0d9ecb6a6bf506e0273ebf7a028"
server_key_iv: "3e0baa71b6eab778a4d6b3e5d5014b6a"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.954236 18799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:26:46.956646 18818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.957950 18799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:26:46.958091 18799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "a105df100caa4f4e98b06a53ad4266ae"
format_stamp: "Formatted at 2026-08-07 02:26:46 on dist-test-slave-g170"
server_key: "fb9ff0d9ecb6a6bf506e0273ebf7a028"
server_key_iv: "3e0baa71b6eab778a4d6b3e5d5014b6a"
server_key_version: "encryptionkey@0"
I20260807 02:26:46.958201 18799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:46.977330 18799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:46.980861 18799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:46.981079 18799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:46.981711 18799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:46.982589 18799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:46.982661 18799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.982730 18799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:46.982781 18799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:46.992040 18799 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:43067
I20260807 02:26:46.992066 18987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:43067 every 8 connection(s)
I20260807 02:26:46.993189 18799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:26:46.998150 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18799
I20260807 02:26:46.998273 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:26:46.998585 17620 external_mini_cluster.cc:1428] Setting key d1b5daf3c69c8c957a442859c1dd8a02
Aug 07 02:26:46 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:47.009903 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:46.995299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:46113 (local address 127.17.53.62:37847)
0807 02:26:46.995484 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:46.995489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:46.996430 (+ 941us) server_negotiation.cc:407] Connection header received
0807 02:26:46.997811 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:46.997815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:46.997888 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:46.997969 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:46.999538 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.000277 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.002078 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.002318 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.002463 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.005405 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.005412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.005418 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.005456 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.007388 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.008009 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.008018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.008022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.008130 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.008494 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.008499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.008501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.008799 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.008921 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.009542 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.009678 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:26:47.010788 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:46.995759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37847 (local address 127.17.53.2:46113)
0807 02:26:46.996244 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:46.996312 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:46.997571 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:46.998191 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:46.998200 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:46.998755 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:46.999332 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:46.999371 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.000418 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.000422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.001929 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.001944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.002508 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.002512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.002699 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.003471 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:47.003488 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:47.005204 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:47.007524 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:47.007530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:47.007545 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:47.007880 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:47.008230 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:47.008232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:47.008233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:47.008381 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:47.008935 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:47.008942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:47.009217 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.009402 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.009690 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":131,"threads_started":1}
I20260807 02:26:47.012167 18990 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37847
I20260807 02:26:47.012461 18990 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:47.013067 18990 heartbeater.cc:511] Master 127.17.53.62:37847 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:26:47.014393 18496 ts_manager.cc:194] Registered new tserver with Master: a105df100caa4f4e98b06a53ad4266ae (127.17.53.2:43067)
I20260807 02:26:47.015261 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:46113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:26:47.031594 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:47.145788 19001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:47.146036 19001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:47.146080 19001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:47.150069 19001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:47.150130 19001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:47.150188 19001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:26:47.155437 19001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37847
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:47.156472 19001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:47.157496 19001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:47.167366 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:47.167323 19009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:47.167403 19008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:47.167465 19001 server_base.cc:1034] running on GCE node
I20260807 02:26:47.167981 19001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:47.168675 19001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:47.169873 19001 hybrid_clock.cc:648] HybridClock initialized: now 1786069607169855 us; error 27 us; skew 500 ppm
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:47.173004 19001 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:26:47.173848 19001 webserver.cc:460] Webserver started at http://127.17.53.3:38339/ using document root <none> and password file <none>
I20260807 02:26:47.174410 19001 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:47.174471 19001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:47.174717 19001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:47.176733 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "8fc4d72f240e4c8b84cd718f1544c60a"
format_stamp: "Formatted at 2026-08-07 02:26:47 on dist-test-slave-g170"
server_key: "59beca1ef34a2301f246adeb63d36d8c"
server_key_iv: "149a00ad632c013fe01c2d823682e2ff"
server_key_version: "encryptionkey@0"
I20260807 02:26:47.177363 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "8fc4d72f240e4c8b84cd718f1544c60a"
format_stamp: "Formatted at 2026-08-07 02:26:47 on dist-test-slave-g170"
server_key: "59beca1ef34a2301f246adeb63d36d8c"
server_key_iv: "149a00ad632c013fe01c2d823682e2ff"
server_key_version: "encryptionkey@0"
I20260807 02:26:47.181057 19001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:47.183418 19026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:47.184528 19001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:47.184667 19001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "8fc4d72f240e4c8b84cd718f1544c60a"
format_stamp: "Formatted at 2026-08-07 02:26:47 on dist-test-slave-g170"
server_key: "59beca1ef34a2301f246adeb63d36d8c"
server_key_iv: "149a00ad632c013fe01c2d823682e2ff"
server_key_version: "encryptionkey@0"
I20260807 02:26:47.184794 19001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:47.206741 19001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:47.210150 19001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:47.210364 19001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:47.210975 19001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:47.211824 19001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:47.211896 19001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:47.211969 19001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:47.212015 19001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:47.221261 19001 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:46279
I20260807 02:26:47.221336 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:46279 every 8 connection(s)
I20260807 02:26:47.222208 19001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:26:47.229194 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19001
I20260807 02:26:47.229364 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:26:47.229722 17620 external_mini_cluster.cc:1428] Setting key 7394e034d960092bd86c87c149f947a6
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:47.236617 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.224165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:47083 (local address 127.17.53.62:37847)
0807 02:26:47.224300 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.224303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.225212 (+ 909us) server_negotiation.cc:407] Connection header received
0807 02:26:47.226606 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.226609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.226669 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.226760 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:47.227956 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.228494 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.229800 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.230051 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.230165 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.233050 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.233053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.233056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.233080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.234553 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.235055 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.235058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.235059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.235181 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.235545 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.235548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.235549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.235721 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.235797 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.236404 (+ 607us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.236501 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 02:26:47.237499 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.224528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37847 (local address 127.17.53.3:47083)
0807 02:26:47.225065 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.225109 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.226389 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.226957 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.226964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.227331 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:47.227807 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.227823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.228622 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.228626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.229675 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.229686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.230246 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.230251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.230453 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.231188 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:47.231211 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:47.232885 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:47.234664 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:47.234669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:47.234680 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:47.234953 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:47.235311 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:47.235314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:47.235315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:47.235453 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:47.235846 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:47.235852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:47.236083 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.236323 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.236622 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":140,"threads_started":1}
I20260807 02:26:47.238665 19185 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37847
I20260807 02:26:47.238967 19185 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:47.239498 19185 heartbeater.cc:511] Master 127.17.53.62:37847 requested a full tablet report, sending...
I20260807 02:26:47.240546 18496 ts_manager.cc:194] Registered new tserver with Master: 8fc4d72f240e4c8b84cd718f1544c60a (127.17.53.3:46279)
I20260807 02:26:47.241298 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:47083
I20260807 02:26:47.243716 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:26:47.254562 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.247419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.17.53.62:37847)
0807 02:26:47.247535 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.247538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.247751 (+ 213us) server_negotiation.cc:407] Connection header received
0807 02:26:47.247842 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.247844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.247881 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.247946 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:47.248626 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.249191 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.250045 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.250247 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.250323 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.251615 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.251618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.251620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.251642 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.253182 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.253547 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.253549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.253550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.253623 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.253866 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.253868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.253869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.254067 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.254167 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.254357 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.254447 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:26:47.259517 18496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:47.262969 18496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:47.278962 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.272517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.17.53.2:43067)
0807 02:26:47.274691 (+ 2174us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.274702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.274723 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:26:47.274799 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.274816 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.275002 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.275148 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:47.275304 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.276208 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.277636 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278201 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.278329 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.278506 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.278668 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.278778 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2064,"thread_start_us":144,"threads_started":1}
I20260807 02:26:47.278975 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.272518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36507 (local address 127.0.0.1:57190)
0807 02:26:47.273358 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.273372 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.273463 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.274130 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.274135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.274154 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:47.274325 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.274331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.275823 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.275827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.277318 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.277329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278491 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.278498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.278589 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.278612 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.278662 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.278789 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":27,"thread_start_us":139,"threads_started":1}
I20260807 02:26:47.279152 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.273119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:46279 (local address 127.0.0.1:36616)
0807 02:26:47.274300 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.274312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.274393 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.274840 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.274843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.274862 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:47.274956 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.274962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.276104 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.276107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.277531 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.277540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278800 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.278804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.278888 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.278909 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.278978 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.279045 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"mutex_wait_us":23,"thread_start_us":107,"threads_started":1}
I20260807 02:26:47.279517 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.273178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.17.53.1:36507)
0807 02:26:47.273602 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.273616 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.273636 (+ 20us) server_negotiation.cc:407] Connection header received
0807 02:26:47.273703 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.273707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.273887 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.274014 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:47.274485 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.275674 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.277459 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278323 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.279042 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.279185 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.279269 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.279366 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":134,"threads_started":1}
I20260807 02:26:47.279728 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.273445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.17.53.3:46279)
0807 02:26:47.273809 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.273821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.274415 (+ 594us) server_negotiation.cc:407] Connection header received
0807 02:26:47.274530 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.274534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.274675 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.274868 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:47.275064 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.275948 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.277662 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278471 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.278566 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.278739 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.279338 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.279422 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":170,"threads_started":1}
I20260807 02:26:47.278955 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.272343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:43067 (local address 127.0.0.1:54390)
0807 02:26:47.273065 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.273117 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.273265 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.275123 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.275126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.275145 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:47.275226 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.275232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.276356 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.276360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.277509 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.277519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.278337 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.278341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.278459 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.278497 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.278635 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.278798 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":128,"thread_start_us":76,"threads_started":1}
I20260807 02:26:47.282516 18691 tablet_service.cc:1467] Processing CreateTablet for tablet 73c99ceb694a4d25912740bd8ee2312a (DEFAULT_TABLE table=test-table [id=e6aabbe7069f4f9ebe001daca5dccef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:47.282655 19087 tablet_service.cc:1467] Processing CreateTablet for tablet 73c99ceb694a4d25912740bd8ee2312a (DEFAULT_TABLE table=test-table [id=e6aabbe7069f4f9ebe001daca5dccef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:47.283031 18891 tablet_service.cc:1467] Processing CreateTablet for tablet 73c99ceb694a4d25912740bd8ee2312a (DEFAULT_TABLE table=test-table [id=e6aabbe7069f4f9ebe001daca5dccef7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:47.283635 18691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c99ceb694a4d25912740bd8ee2312a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:47.283744 19087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c99ceb694a4d25912740bd8ee2312a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:47.283982 18891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c99ceb694a4d25912740bd8ee2312a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:47.291465 19208 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Bootstrap starting.
I20260807 02:26:47.291464 19209 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Bootstrap starting.
I20260807 02:26:47.294054 19209 tablet_bootstrap.cc:654] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:47.294597 19208 tablet_bootstrap.cc:654] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:47.295490 19208 log.cc:824] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:47.295490 19209 log.cc:824] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:47.296304 19206 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Bootstrap starting.
I20260807 02:26:47.297915 19209 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: No bootstrap required, opened a new log
I20260807 02:26:47.298126 19209 ts_tablet_manager.cc:1397] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 02:26:47.298460 19206 tablet_bootstrap.cc:654] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:47.298599 19208 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: No bootstrap required, opened a new log
I20260807 02:26:47.298784 19208 ts_tablet_manager.cc:1397] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 02:26:47.299108 19206 log.cc:824] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:47.300719 19206 tablet_bootstrap.cc:492] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: No bootstrap required, opened a new log
I20260807 02:26:47.300926 19206 ts_tablet_manager.cc:1397] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 02:26:47.302240 19209 raft_consensus.cc:348] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.302538 19209 raft_consensus.cc:374] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:47.302590 19209 raft_consensus.cc:729] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc4d72f240e4c8b84cd718f1544c60a, State: Initialized, Role: FOLLOWER
I20260807 02:26:47.302647 19208 raft_consensus.cc:348] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.302907 19208 raft_consensus.cc:374] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:47.302970 19208 raft_consensus.cc:729] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a105df100caa4f4e98b06a53ad4266ae, State: Initialized, Role: FOLLOWER
I20260807 02:26:47.303349 19208 consensus_queue.cc:260] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.303349 19209 consensus_queue.cc:260] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.304066 18990 heartbeater.cc:503] Master 127.17.53.62:37847 was elected leader, sending a full tablet report...
I20260807 02:26:47.304389 19208 ts_tablet_manager.cc:1428] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:26:47.304389 19209 ts_tablet_manager.cc:1428] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:26:47.304968 19185 heartbeater.cc:503] Master 127.17.53.62:37847 was elected leader, sending a full tablet report...
I20260807 02:26:47.304957 19206 raft_consensus.cc:348] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.305205 19206 raft_consensus.cc:374] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:47.305254 19206 raft_consensus.cc:729] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60e68ffc2824cc2823b50a6b1709d5f, State: Initialized, Role: FOLLOWER
I20260807 02:26:47.305626 19206 consensus_queue.cc:260] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.306468 19206 ts_tablet_manager.cc:1428] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 02:26:47.306826 18791 heartbeater.cc:503] Master 127.17.53.62:37847 was elected leader, sending a full tablet report...
W20260807 02:26:47.473204 19187 tablet.cc:2369] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:26:47.494489 18993 tablet.cc:2369] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:47.514297 19216 raft_consensus.cc:482] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:47.514551 19216 raft_consensus.cc:504] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.515805 19216 leader_election.cc:288] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60e68ffc2824cc2823b50a6b1709d5f (127.17.53.1:36507), 8fc4d72f240e4c8b84cd718f1544c60a (127.17.53.3:46279)
W20260807 02:26:47.520109 18792 tablet.cc:2369] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:47.521027 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.516072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:46279 (local address 127.17.53.2:44549)
0807 02:26:47.516432 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.516448 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.516582 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.516915 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.516919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.516953 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:47.517237 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.517243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.518525 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.518530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.519562 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.519573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.520452 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.520456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.520596 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.520625 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.520718 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.520843 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":118,"spinlock_wait_cycles":11520,"thread_start_us":93,"threads_started":1}
I20260807 02:26:47.521044 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.516256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:44549 (local address 127.17.53.3:46279)
0807 02:26:47.516418 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.516424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.516540 (+ 116us) server_negotiation.cc:407] Connection header received
0807 02:26:47.516715 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.516719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.516797 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.516884 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:47.517374 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.518319 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.519732 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.520179 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.520313 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.520393 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.520789 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.520908 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:26:47.521117 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.516265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:41235 (local address 127.17.53.1:36507)
0807 02:26:47.516483 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.516487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.516529 (+ 42us) server_negotiation.cc:407] Connection header received
0807 02:26:47.516716 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.516719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.516802 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.516888 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:47.517374 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.518288 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.519778 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.520411 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.520530 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.520605 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.520912 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.520996 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 02:26:47.521068 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.516081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36507 (local address 127.17.53.2:41235)
0807 02:26:47.516431 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:47.516448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:47.516582 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:47.516915 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:47.516919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:47.516953 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:47.517231 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.517239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.518487 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.518492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.519559 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:47.519571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.520534 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.520537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:47.520681 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.520718 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 02:26:47.520868 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 02:26:47.520935 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":106,"spinlock_wait_cycles":42752,"thread_start_us":111,"threads_started":1}
I20260807 02:26:47.521970 18726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c99ceb694a4d25912740bd8ee2312a" candidate_uuid: "a105df100caa4f4e98b06a53ad4266ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" is_pre_election: true
I20260807 02:26:47.522083 19119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c99ceb694a4d25912740bd8ee2312a" candidate_uuid: "a105df100caa4f4e98b06a53ad4266ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" is_pre_election: true
I20260807 02:26:47.522372 18726 raft_consensus.cc:2454] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a105df100caa4f4e98b06a53ad4266ae in term 0.
I20260807 02:26:47.522429 19119 raft_consensus.cc:2454] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a105df100caa4f4e98b06a53ad4266ae in term 0.
I20260807 02:26:47.522935 18827 leader_election.cc:302] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a105df100caa4f4e98b06a53ad4266ae, b60e68ffc2824cc2823b50a6b1709d5f; no voters:
I20260807 02:26:47.523267 19216 raft_consensus.cc:2781] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:47.523384 19216 raft_consensus.cc:482] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:47.523438 19216 raft_consensus.cc:3037] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:47.524566 19216 raft_consensus.cc:504] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.525054 19216 leader_election.cc:288] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [CANDIDATE]: Term 1 election: Requested vote from peers b60e68ffc2824cc2823b50a6b1709d5f (127.17.53.1:36507), 8fc4d72f240e4c8b84cd718f1544c60a (127.17.53.3:46279)
I20260807 02:26:47.525520 19119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c99ceb694a4d25912740bd8ee2312a" candidate_uuid: "a105df100caa4f4e98b06a53ad4266ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc4d72f240e4c8b84cd718f1544c60a"
I20260807 02:26:47.525527 18726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c99ceb694a4d25912740bd8ee2312a" candidate_uuid: "a105df100caa4f4e98b06a53ad4266ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60e68ffc2824cc2823b50a6b1709d5f"
I20260807 02:26:47.525666 19119 raft_consensus.cc:3037] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:47.525691 18726 raft_consensus.cc:3037] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:47.526803 19119 raft_consensus.cc:2454] T 73c99ceb694a4d25912740bd8ee2312a P 8fc4d72f240e4c8b84cd718f1544c60a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a105df100caa4f4e98b06a53ad4266ae in term 1.
I20260807 02:26:47.526829 18726 raft_consensus.cc:2454] T 73c99ceb694a4d25912740bd8ee2312a P b60e68ffc2824cc2823b50a6b1709d5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a105df100caa4f4e98b06a53ad4266ae in term 1.
I20260807 02:26:47.527153 18822 leader_election.cc:302] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fc4d72f240e4c8b84cd718f1544c60a, a105df100caa4f4e98b06a53ad4266ae; no voters:
I20260807 02:26:47.527359 19216 raft_consensus.cc:2781] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:47.527590 19216 raft_consensus.cc:686] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [term 1 LEADER]: Becoming Leader. State: Replica: a105df100caa4f4e98b06a53ad4266ae, State: Running, Role: LEADER
I20260807 02:26:47.527957 19216 consensus_queue.cc:237] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } }
I20260807 02:26:47.531199 18495 catalog_manager.cc:5495] T 73c99ceb694a4d25912740bd8ee2312a P a105df100caa4f4e98b06a53ad4266ae reported cstate change: term changed from 0 to 1, leader changed from <none> to a105df100caa4f4e98b06a53ad4266ae (127.17.53.2). New cstate: current_term: 1 leader_uuid: "a105df100caa4f4e98b06a53ad4266ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a105df100caa4f4e98b06a53ad4266ae" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 43067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60e68ffc2824cc2823b50a6b1709d5f" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fc4d72f240e4c8b84cd718f1544c60a" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46279 } health_report { overall_health: UNKNOWN } } }
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:47.576169 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.565444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.17.53.1:36507)
0807 02:26:47.565653 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.565659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.565868 (+ 209us) server_negotiation.cc:407] Connection header received
0807 02:26:47.566047 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.566051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.566107 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.566234 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:47.567017 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.567572 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.568524 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.568831 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.568958 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.571873 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.571891 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.571910 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.571938 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.573991 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.574578 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.574585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.574587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.574678 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.574996 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.575000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.575002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.575302 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.575597 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.575815 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.575957 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:47.607717 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.598477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.17.53.1:36507)
0807 02:26:47.598675 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.598681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.598839 (+ 158us) server_negotiation.cc:407] Connection header received
0807 02:26:47.599037 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.599040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.599099 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.599194 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:47.599969 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.600556 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.601575 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.601844 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.601964 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.604576 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.604580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.604582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.604607 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.605974 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.606413 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.606416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.606418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.606504 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.606786 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.606791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.606796 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.606976 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.607117 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.607413 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.607546 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[18428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:47.636634 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:47.627340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.17.53.1:36507)
0807 02:26:47.627531 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:47.627537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:47.627681 (+ 144us) server_negotiation.cc:407] Connection header received
0807 02:26:47.627798 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:47.627801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:47.627851 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:47.627934 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:47.628606 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.629212 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.630079 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:47.630322 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:47.630417 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:47.633126 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:47.633129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:47.633131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:47.633158 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:47.634814 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.635291 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.635294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.635295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.635387 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:47.635722 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:47.635726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:47.635727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:47.635919 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:47.636035 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:47.636270 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 02:26:47.636414 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 02:26:47.638224 18650 tablet_service.cc:2908] Rejecting scan request for tablet 73c99ceb694a4d25912740bd8ee2312a: Uninitialized: clean time has not yet been initialized
I20260807 02:26:47.640935 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18595
I20260807 02:26:47.660461 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18799
I20260807 02:26:47.672616 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19001
I20260807 02:26:47.680101 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18444
2026-08-07T02:26:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2837 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[19250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[19250](info): set up 2 sockets
Aug 07 02:26:47 dist-test-slave-g170 krb5kdc[19250](info): commencing operation
krb5kdc: starting...
W20260807 02:26:48.930364 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:26:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:26:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:26:49.068149 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:43151
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:43151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:49.185359 19271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:49.185644 19271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:49.185688 19271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:49.189792 19271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:26:49.189855 19271 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 02:26:49.189873 19271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:49.189896 19271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:49.189913 19271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:26:49.189929 19271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:26:49.195267 19271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:43151
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:43151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:49.196336 19271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:49.197444 19271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:49.206154 19280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:49.206192 19278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:49.206192 19283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:49.206321 19271 server_base.cc:1034] running on GCE node
I20260807 02:26:49.207091 19271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:49.208199 19271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:49.209380 19271 hybrid_clock.cc:648] HybridClock initialized: now 1786069609209350 us; error 43 us; skew 500 ppm
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:49.216203 19271 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:26:49.217232 19271 webserver.cc:460] Webserver started at http://127.17.53.62:35085/ using document root <none> and password file <none>
I20260807 02:26:49.217855 19271 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:49.217955 19271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:49.218199 19271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:49.220323 19271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "00ea9d8e40a84a18a2dfd378582dbfe1"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "efbdaec615ee3dc0f36de9a172c1f780"
server_key_iv: "3d3f5d762b20971315a8088258e6047b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.220984 19271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "00ea9d8e40a84a18a2dfd378582dbfe1"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "efbdaec615ee3dc0f36de9a172c1f780"
server_key_iv: "3d3f5d762b20971315a8088258e6047b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.225078 19271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:26:49.227739 19291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:49.229153 19271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:49.229302 19271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "00ea9d8e40a84a18a2dfd378582dbfe1"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "efbdaec615ee3dc0f36de9a172c1f780"
server_key_iv: "3d3f5d762b20971315a8088258e6047b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.229426 19271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:49.248570 19271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:49.251899 19271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:49.252123 19271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:49.261149 19271 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:43151
I20260807 02:26:49.261188 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:43151 every 8 connection(s)
I20260807 02:26:49.262142 19271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:26:49.264967 19384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:49.266012 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19271
I20260807 02:26:49.266135 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:26:49.266439 17620 external_mini_cluster.cc:1428] Setting key c59784ec3fc417ead947c38b58ebddaa
I20260807 02:26:49.271333 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Bootstrap starting.
I20260807 02:26:49.274513 19384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:49.275257 19384 log.cc:824] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:49.277724 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: No bootstrap required, opened a new log
I20260807 02:26:49.281002 19384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } }
I20260807 02:26:49.281266 19384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:49.281337 19384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ea9d8e40a84a18a2dfd378582dbfe1, State: Initialized, Role: FOLLOWER
I20260807 02:26:49.281849 19384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } }
I20260807 02:26:49.282032 19384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:26:49.282167 19384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:26:49.282294 19384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:49.283212 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.268002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.17.53.62:43151)
0807 02:26:49.268495 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:49.268510 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:49.268548 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:26:49.269290 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:49.269310 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:49.269659 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:49.269992 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:49.270815 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.272411 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.273672 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.274285 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.274473 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.277506 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:49.277512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:49.277525 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:49.277552 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:49.280041 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.280572 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.280577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.280582 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.280698 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.281011 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.281013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.281014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.281501 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:49.281710 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:49.281949 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 02:26:49.282474 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":163,"threads_started":1}
I20260807 02:26:49.283205 19384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } }
I20260807 02:26:49.283670 19384 leader_election.cc:302] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ea9d8e40a84a18a2dfd378582dbfe1; no voters:
I20260807 02:26:49.284173 19384 leader_election.cc:288] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:26:49.284281 19392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:49.284472 19392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [term 1 LEADER]: Becoming Leader. State: Replica: 00ea9d8e40a84a18a2dfd378582dbfe1, State: Running, Role: LEADER
I20260807 02:26:49.284864 19392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } }
I20260807 02:26:49.285564 19384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:26:49.286592 19393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } } }
I20260807 02:26:49.287032 19393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:49.286928 19395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ea9d8e40a84a18a2dfd378582dbfe1. Latest consensus state: current_term: 1 leader_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea9d8e40a84a18a2dfd378582dbfe1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43151 } } }
I20260807 02:26:49.287374 19395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1 [sys.catalog]: This master's current role is: LEADER
W20260807 02:26:49.291006 19419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:26:49.291088 19419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:26:49.291158 19407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:26:49.291831 19407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:26:49.297736 19407 catalog_manager.cc:1347] Generated new cluster ID: 02659539feac4b2a94e2a821f781c16e
I20260807 02:26:49.297816 19407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:26:49.316650 19407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:26:49.318100 19407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:26:49.324445 19407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Generated new TSK 0
I20260807 02:26:49.325377 19407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:26:49.367926 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:43151
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:49.489312 19424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:49.489539 19424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:49.489581 19424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:49.493489 19424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:49.493536 19424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:49.493573 19424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:49.493666 19424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:26:49.498766 19424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:49.499768 19424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:49.500849 19424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:49.510751 19432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:49.510865 19424 server_base.cc:1034] running on GCE node
W20260807 02:26:49.510742 19431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:49.510814 19437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:49.511569 19424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:49.512243 19424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:49.513441 19424 hybrid_clock.cc:648] HybridClock initialized: now 1786069609513426 us; error 26 us; skew 500 ppm
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:49.516207 19424 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:26:49.517117 19424 webserver.cc:460] Webserver started at http://127.17.53.1:36479/ using document root <none> and password file <none>
I20260807 02:26:49.517688 19424 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:49.517820 19424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:49.518059 19424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:49.520128 19424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "4ec319e391af4f5ea85980bd8f69b445"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "8e334b74e088951782f4a32af7e3ba4f"
server_key_iv: "44fa70faf86dfb94649c803ece69aa80"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.520701 19424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "4ec319e391af4f5ea85980bd8f69b445"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "8e334b74e088951782f4a32af7e3ba4f"
server_key_iv: "44fa70faf86dfb94649c803ece69aa80"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.524538 19424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 02:26:49.527086 19445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:49.528304 19424 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:49.528441 19424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "4ec319e391af4f5ea85980bd8f69b445"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "8e334b74e088951782f4a32af7e3ba4f"
server_key_iv: "44fa70faf86dfb94649c803ece69aa80"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.528551 19424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:49.548388 19424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:49.551156 19424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:49.551355 19424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:49.552739 19424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:49.552873 19424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:49.552944 19424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:49.552996 19424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:49.565004 19424 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:41949
I20260807 02:26:49.565086 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:41949 every 8 connection(s)
I20260807 02:26:49.566123 19424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:26:49.567651 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.553683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:38901 (local address 127.17.53.62:43151)
0807 02:26:49.553817 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:49.553821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:49.554874 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 02:26:49.555815 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:49.555818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:49.555879 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:49.555992 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:49.557457 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.558117 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.559493 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.559733 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.559849 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.563737 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:49.563744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:49.563750 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:49.563783 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:49.565517 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.566002 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.566006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.566007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.566105 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.566490 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.566493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.566495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.566735 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:49.566848 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:49.567422 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 02:26:49.567529 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:26:49.568423 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.554204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.1:38901)
0807 02:26:49.554730 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:49.554774 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:49.555615 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:49.556291 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:49.556301 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:49.556643 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:49.557309 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.557325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.558254 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.558258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.559366 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.559377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.559891 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.559896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.560070 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.560722 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:49.560747 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:49.563550 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:49.565627 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.565632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.565642 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.565895 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.566233 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.566236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.566238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.566384 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.566909 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:49.566915 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:49.567196 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 02:26:49.567358 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 02:26:49.567591 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":133,"threads_started":1}
I20260807 02:26:49.569759 19624 heartbeater.cc:347] Connected to a master server at 127.17.53.62:43151
I20260807 02:26:49.569898 19624 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:49.570433 19624 heartbeater.cc:511] Master 127.17.53.62:43151 requested a full tablet report, sending...
I20260807 02:26:49.571972 19322 ts_manager.cc:194] Registered new tserver with Master: 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949)
I20260807 02:26:49.573546 19322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:38901
I20260807 02:26:49.575810 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19424
I20260807 02:26:49.575955 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:26:49.576318 17620 external_mini_cluster.cc:1428] Setting key a419615ecaa2bf3da8de8900ddc99065
I20260807 02:26:49.581638 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.573549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:45469 (local address 127.17.53.62:43151)
0807 02:26:49.573725 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:49.573730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:49.573747 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:26:49.573787 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:49.573791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:49.573832 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:49.573935 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:49.574414 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.574986 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.576092 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.576317 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.576428 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.577950 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:49.577955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:49.577958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:49.577986 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:49.579782 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.580216 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.580222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.580227 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.580334 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.580711 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.580716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.580719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.580994 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:49.581079 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:49.581416 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 02:26:49.581520 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 02:26:49.581861 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.572033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.1:45469)
0807 02:26:49.572316 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:49.572328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:49.572396 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:49.573956 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:49.573958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:49.574129 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:49.574281 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.574289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.575127 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.575130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.575969 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.575979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.576435 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.576438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.576535 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.577233 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:49.577250 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:49.577791 (+ 541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:49.579907 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.579909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.579911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.580112 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.580463 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.580466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.580467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.580560 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.581098 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:49.581104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:49.581228 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 02:26:49.581359 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 02:26:49.581546 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:26:49.607997 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:43151
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:49.730280 19637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:49.730590 19637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:49.730640 19637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:49.735491 19637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:49.735572 19637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:49.735616 19637 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:49.735683 19637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:26:49.741710 19637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:49.742951 19637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:49.744086 19637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:49.755088 19647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:49.755165 19648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:49.755168 19650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:49.755335 19637 server_base.cc:1034] running on GCE node
I20260807 02:26:49.755864 19637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:49.756592 19637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:49.757777 19637 hybrid_clock.cc:648] HybridClock initialized: now 1786069609757739 us; error 54 us; skew 500 ppm
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:49.760879 19637 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:26:49.761945 19637 webserver.cc:460] Webserver started at http://127.17.53.2:34621/ using document root <none> and password file <none>
I20260807 02:26:49.762547 19637 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:49.762614 19637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:49.762801 19637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:49.765193 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "03a739030832483a9926c0ae183ae1a7"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "649540cc392240eac46ff64fa2d195fe"
server_key_iv: "606a75b8d3c6f1033c480a49cbe80e5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.765803 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "03a739030832483a9926c0ae183ae1a7"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "649540cc392240eac46ff64fa2d195fe"
server_key_iv: "606a75b8d3c6f1033c480a49cbe80e5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.770136 19637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:26:49.773545 19662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:49.775175 19637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:26:49.775333 19637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "03a739030832483a9926c0ae183ae1a7"
format_stamp: "Formatted at 2026-08-07 02:26:49 on dist-test-slave-g170"
server_key: "649540cc392240eac46ff64fa2d195fe"
server_key_iv: "606a75b8d3c6f1033c480a49cbe80e5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:49.775475 19637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:49.805114 19637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:49.808517 19637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:49.808748 19637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:49.810433 19637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:49.810485 19637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:49.810529 19637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:49.810583 19637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:49 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:49.822441 19637 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:33889
I20260807 02:26:49.822544 19835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:33889 every 8 connection(s)
I20260807 02:26:49.823472 19637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:26:49.826570 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19637
I20260807 02:26:49.826676 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:26:49.827044 17620 external_mini_cluster.cc:1428] Setting key 4ebf6ae613086ac0ee45dc6588fbbfd4
I20260807 02:26:49.827471 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.811563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:37707 (local address 127.17.53.62:43151)
0807 02:26:49.811702 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:49.811705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:49.812905 (+ 1200us) server_negotiation.cc:407] Connection header received
0807 02:26:49.814013 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:49.814016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:49.814079 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:49.814203 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:49.815954 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.816867 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.818835 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.819145 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.819231 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.822819 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:49.822823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:49.822825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:49.822854 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:49.824751 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.825427 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.825430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.825432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.825547 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.825980 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.825983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.825984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.826290 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:49.826416 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:49.827055 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 02:26:49.827224 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:26:49.828239 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.812149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.2:37707)
0807 02:26:49.812681 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:49.812725 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:49.813759 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:49.814454 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:49.814467 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:49.814966 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:49.815755 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.815779 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.817026 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.817030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.818671 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.818685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.819336 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.819341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.819568 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.820356 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:49.820387 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:49.822586 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:49.824956 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.824963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.824985 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.825266 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.825679 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.825682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.825683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.825874 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.826402 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:49.826408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:49.826752 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 02:26:49.826951 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 02:26:49.827303 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":152,"threads_started":1}
I20260807 02:26:49.829710 19836 heartbeater.cc:347] Connected to a master server at 127.17.53.62:43151
I20260807 02:26:49.829937 19836 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:49.830585 19836 heartbeater.cc:511] Master 127.17.53.62:43151 requested a full tablet report, sending...
I20260807 02:26:49.831817 19322 ts_manager.cc:194] Registered new tserver with Master: 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889)
I20260807 02:26:49.832497 19322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:37707
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:26:49.840083 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.832843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60247 (local address 127.17.53.62:43151)
0807 02:26:49.832980 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:49.832985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:49.833070 (+ 85us) server_negotiation.cc:407] Connection header received
0807 02:26:49.833221 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:49.833224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:49.833272 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:49.833359 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:49.833880 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.834375 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.835210 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.835427 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.835528 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.836747 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:49.836750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:49.836823 (+ 73us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:49.836855 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:49.838279 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.838814 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.838820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.838826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.838899 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:49.839183 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:49.839187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:49.839189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:49.839411 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:49.839524 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:49.839797 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 02:26:49.839909 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:26:49.840104 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:49.832306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.2:60247)
0807 02:26:49.832998 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:49.833014 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:49.833117 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:49.833379 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:49.833382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:49.833604 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:49.833778 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.833786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.834503 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.834507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.835080 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:49.835088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:49.835544 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:49.835592 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:49.835701 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:49.836253 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:49.836265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:49.836598 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:49.838440 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.838444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.838446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.838700 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.838993 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:49.838997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:49.838999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:49.839088 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:49.839514 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:49.839518 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:49.839620 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 02:26:49.839730 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 02:26:49.839913 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":2,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:26:49.860723 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:43151
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:49.978749 19847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:49.979040 19847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:49.979131 19847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:49.983310 19847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:49.983366 19847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:49.983405 19847 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:49.983485 19847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:26:49.989116 19847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:43151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:49.990182 19847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:49.991214 19847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:50.001458 19855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:50.001458 19859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:50.001473 19854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:50.001842 19847 server_base.cc:1034] running on GCE node
I20260807 02:26:50.002358 19847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:50.003094 19847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:50.004333 19847 hybrid_clock.cc:648] HybridClock initialized: now 1786069610004312 us; error 43 us; skew 500 ppm
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069610, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:50.007727 19847 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:26:50.008862 19847 webserver.cc:460] Webserver started at http://127.17.53.3:43515/ using document root <none> and password file <none>
I20260807 02:26:50.009512 19847 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:50.009620 19847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:50.009913 19847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:50.012145 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c"
format_stamp: "Formatted at 2026-08-07 02:26:50 on dist-test-slave-g170"
server_key: "2c59d8eedf24c0c2508cda82ff63db1e"
server_key_iv: "57409e045817bdeca402d00ebc5d0719"
server_key_version: "encryptionkey@0"
I20260807 02:26:50.012830 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c"
format_stamp: "Formatted at 2026-08-07 02:26:50 on dist-test-slave-g170"
server_key: "2c59d8eedf24c0c2508cda82ff63db1e"
server_key_iv: "57409e045817bdeca402d00ebc5d0719"
server_key_version: "encryptionkey@0"
I20260807 02:26:50.017069 19847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:26:50.019963 19869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:50.021416 19847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:50.021575 19847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c"
format_stamp: "Formatted at 2026-08-07 02:26:50 on dist-test-slave-g170"
server_key: "2c59d8eedf24c0c2508cda82ff63db1e"
server_key_iv: "57409e045817bdeca402d00ebc5d0719"
server_key_version: "encryptionkey@0"
I20260807 02:26:50.021695 19847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:50.040143 19847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:50.043303 19847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:50.043519 19847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:50.045104 19847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:50.045178 19847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:50.045251 19847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:50.045305 19847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069610, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:50.056695 19847 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:33281
I20260807 02:26:50.056886 20041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:33281 every 8 connection(s)
I20260807 02:26:50.057719 19847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:26:50.058609 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19847
I20260807 02:26:50.058745 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:26:50.059037 17620 external_mini_cluster.cc:1428] Setting key 0673f2c4f50eeae87aa6f0a8d549f134
I20260807 02:26:50.060849 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.046176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50971 (local address 127.17.53.62:43151)
0807 02:26:50.046338 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.046344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.047299 (+ 955us) server_negotiation.cc:407] Connection header received
0807 02:26:50.048683 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.048687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.048810 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.048926 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.050455 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.051071 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.052394 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.052848 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.052984 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.056006 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.056013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.056016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.056047 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.058097 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.058798 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.058802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.058804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.058917 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.059309 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.059314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.059316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.059606 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.059716 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.060523 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.060656 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 02:26:50.061786 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.046575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.3:50971)
0807 02:26:50.047103 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.047170 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.048382 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.049135 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.049143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.049631 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:50.050285 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.050301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.051225 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.051229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.052222 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.052233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.053064 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.053070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.053327 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.054020 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:50.054038 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:50.055815 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:50.058236 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.058243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.058260 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.058661 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.059040 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.059043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.059045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.059195 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.059883 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:50.059890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:50.060205 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.060448 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.060726 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":157,"threads_started":1}
I20260807 02:26:50.063091 20044 heartbeater.cc:347] Connected to a master server at 127.17.53.62:43151
I20260807 02:26:50.063241 20044 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:50.063831 20044 heartbeater.cc:511] Master 127.17.53.62:43151 requested a full tablet report, sending...
I20260807 02:26:50.065201 19320 ts_manager.cc:194] Registered new tserver with Master: a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281)
I20260807 02:26:50.065995 19320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:50971
I20260807 02:26:50.073065 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.065785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:47957 (local address 127.17.53.62:43151)
0807 02:26:50.065913 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.065918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.066175 (+ 257us) server_negotiation.cc:407] Connection header received
0807 02:26:50.066381 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.066385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.066434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.066518 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.066985 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.067517 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.068350 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.068584 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.068693 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.069856 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.069865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.069868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.069933 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.071341 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.071799 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.071802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.071804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.071891 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.072197 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.072200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.072202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.072388 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.072502 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.072799 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.072923 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:26:50.073079 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.065656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.17.53.3:47957)
0807 02:26:50.066091 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.066108 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.066246 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.066536 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.066539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.066757 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:50.066868 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.066874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.067647 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.067651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.068216 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.068224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.068698 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.068702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.068831 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.069347 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:50.069359 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:50.069674 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:50.071453 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.071457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.071459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.071685 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.071984 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.071988 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.072013 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.072098 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.072490 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:50.072493 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:50.072597 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.072708 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.072876 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":109,"threads_started":1}
I20260807 02:26:50.073421 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:50.102857 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.093888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.17.53.62:43151)
0807 02:26:50.094022 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.094025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.094037 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:26:50.094078 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.094080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.094119 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.094182 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.094945 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.095535 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.096425 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.096690 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.096874 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.099203 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.099207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.099210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.099236 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.100973 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.101451 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.101457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.101458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.101557 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.101882 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.101887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.101889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.102098 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.102257 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.102529 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.102669 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:26:50.106210 19320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:50.109874 19320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:50.125998 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.119220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33281 (local address 127.0.0.1:56252)
0807 02:26:50.120254 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.120270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.120361 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.121289 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.121292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.121313 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.121453 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.121464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.122761 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.122766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.123931 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.123942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125310 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.125452 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.125521 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.125667 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.125819 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":3,"thread_start_us":122,"threads_started":1}
I20260807 02:26:50.126055 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.118876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:33889 (local address 127.0.0.1:50008)
0807 02:26:50.119334 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.119391 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.119527 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.120463 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.120469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.120493 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.120625 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.120632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.122478 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.122482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.124060 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.124072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125616 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125648 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.125782 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.125804 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.125875 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.125933 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":90,"thread_start_us":136,"threads_started":1}
I20260807 02:26:50.126055 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.119816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.17.53.3:33281)
0807 02:26:50.120877 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.120891 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.120920 (+ 29us) server_negotiation.cc:407] Connection header received
0807 02:26:50.120987 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.120991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.121140 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.121316 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.121634 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.122596 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.124081 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125123 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125306 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.125502 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.125716 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.125876 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":123,"threads_started":1}
I20260807 02:26:50.126871 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.120292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41949 (local address 127.0.0.1:58102)
0807 02:26:50.120644 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.120661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.120794 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.121495 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.121500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.121530 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.121649 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.121656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.123065 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.123069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.124241 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.124254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125642 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.126532 (+ 885us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.126561 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.126648 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.126728 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260807 02:26:50.126991 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.120714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.17.53.1:41949)
0807 02:26:50.121039 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.121048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.121068 (+ 20us) server_negotiation.cc:407] Connection header received
0807 02:26:50.121150 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.121155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.121345 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.121501 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.121783 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.122915 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.124394 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125132 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125280 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.125465 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.126698 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.126825 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":145,"threads_started":1}
I20260807 02:26:50.127817 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.119173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.17.53.2:33889)
0807 02:26:50.119867 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.119882 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.119914 (+ 32us) server_negotiation.cc:407] Connection header received
0807 02:26:50.119996 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.120002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.120241 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.120459 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.121069 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.122329 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.124207 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.125127 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.125306 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.125502 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.127580 (+ 2078us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.127685 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":129,"threads_started":1}
I20260807 02:26:50.130160 19944 tablet_service.cc:1467] Processing CreateTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 (DEFAULT_TABLE table=test-table [id=facecdc2d55f4e64856e96d1baad4abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.130300 19515 tablet_service.cc:1467] Processing CreateTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 (DEFAULT_TABLE table=test-table [id=facecdc2d55f4e64856e96d1baad4abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.130690 19732 tablet_service.cc:1467] Processing CreateTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 (DEFAULT_TABLE table=test-table [id=facecdc2d55f4e64856e96d1baad4abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.131580 19515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c2f13c2d74a81bc933635fc5e1f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.131637 19944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c2f13c2d74a81bc933635fc5e1f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.131731 19732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676c2f13c2d74a81bc933635fc5e1f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.139823 20081 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Bootstrap starting.
I20260807 02:26:50.141235 20085 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Bootstrap starting.
I20260807 02:26:50.143245 20081 tablet_bootstrap.cc:654] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.143281 20085 tablet_bootstrap.cc:654] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.144064 20085 log.cc:824] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:50.144068 20081 log.cc:824] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:50.144855 20083 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Bootstrap starting.
I20260807 02:26:50.146353 20085 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: No bootstrap required, opened a new log
I20260807 02:26:50.146598 20085 ts_tablet_manager.cc:1397] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 02:26:50.146616 20081 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: No bootstrap required, opened a new log
I20260807 02:26:50.146816 20081 ts_tablet_manager.cc:1397] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 02:26:50.147238 20083 tablet_bootstrap.cc:654] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.148281 20083 log.cc:824] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:50.150655 20083 tablet_bootstrap.cc:492] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: No bootstrap required, opened a new log
I20260807 02:26:50.150574 20085 raft_consensus.cc:348] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.150868 20083 ts_tablet_manager.cc:1397] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260807 02:26:50.151216 20085 raft_consensus.cc:374] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.151304 20085 raft_consensus.cc:729] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a739030832483a9926c0ae183ae1a7, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.151825 20085 consensus_queue.cc:260] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.151834 20081 raft_consensus.cc:348] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.152110 20081 raft_consensus.cc:374] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.152168 20081 raft_consensus.cc:729] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec319e391af4f5ea85980bd8f69b445, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.152593 20081 consensus_queue.cc:260] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.153043 20085 ts_tablet_manager.cc:1428] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 02:26:50.153416 19836 heartbeater.cc:503] Master 127.17.53.62:43151 was elected leader, sending a full tablet report...
I20260807 02:26:50.153416 19624 heartbeater.cc:503] Master 127.17.53.62:43151 was elected leader, sending a full tablet report...
I20260807 02:26:50.153601 20081 ts_tablet_manager.cc:1428] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260807 02:26:50.154656 20083 raft_consensus.cc:348] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.154911 20083 raft_consensus.cc:374] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.155011 20083 raft_consensus.cc:729] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77ea3f8d14a48f7a3b30ffd5ac6a85c, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.155418 20083 consensus_queue.cc:260] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.156080 20044 heartbeater.cc:503] Master 127.17.53.62:43151 was elected leader, sending a full tablet report...
I20260807 02:26:50.156378 20083 ts_tablet_manager.cc:1428] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260807 02:26:50.173102 20093 raft_consensus.cc:482] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:50.173305 20093 raft_consensus.cc:504] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.174443 20093 leader_election.cc:288] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281), 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949)
I20260807 02:26:50.178635 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.174817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:38751 (local address 127.17.53.1:41949)
0807 02:26:50.174973 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.174979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.175068 (+ 89us) server_negotiation.cc:407] Connection header received
0807 02:26:50.175246 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.175250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.175298 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.175386 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.175686 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.176264 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.177495 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.177976 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.178124 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.178199 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.178422 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.178508 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:26:50.178644 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.174730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41949 (local address 127.17.53.2:38751)
0807 02:26:50.174962 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.174980 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.175089 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.175411 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.175415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.175435 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.175560 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.175565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.176390 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.176396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.177351 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.177362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.178128 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.178132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.178271 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.178291 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.178402 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.178482 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 02:26:50.179260 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.174940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:49271 (local address 127.17.53.3:33281)
0807 02:26:50.175089 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.175094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.175327 (+ 233us) server_negotiation.cc:407] Connection header received
0807 02:26:50.175494 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.175497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.175550 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.175639 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.176388 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.177001 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.178107 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.178544 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.178673 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.178737 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.178970 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.179054 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 02:26:50.179431 19550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c2f13c2d74a81bc933635fc5e1f64" candidate_uuid: "03a739030832483a9926c0ae183ae1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec319e391af4f5ea85980bd8f69b445" is_pre_election: true
I20260807 02:26:50.179747 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.174735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33281 (local address 127.17.53.2:49271)
0807 02:26:50.175190 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.175203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.175363 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.175672 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.175675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.175693 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.176229 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.176236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.177132 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.177136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.177964 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.177974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.178680 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.178684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.178808 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.178839 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.179438 (+ 599us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.179521 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":35,"thread_start_us":135,"threads_started":1}
I20260807 02:26:50.179795 19550 raft_consensus.cc:2454] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a739030832483a9926c0ae183ae1a7 in term 0.
I20260807 02:26:50.180392 19667 leader_election.cc:302] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a739030832483a9926c0ae183ae1a7, 4ec319e391af4f5ea85980bd8f69b445; no voters:
I20260807 02:26:50.180546 19978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c2f13c2d74a81bc933635fc5e1f64" candidate_uuid: "03a739030832483a9926c0ae183ae1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" is_pre_election: true
I20260807 02:26:50.180725 20093 raft_consensus.cc:2781] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:50.180835 20093 raft_consensus.cc:482] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:50.180910 20093 raft_consensus.cc:3037] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.180876 19978 raft_consensus.cc:2454] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a739030832483a9926c0ae183ae1a7 in term 0.
I20260807 02:26:50.182158 20093 raft_consensus.cc:504] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.182559 20093 leader_election.cc:288] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [CANDIDATE]: Term 1 election: Requested vote from peers a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281), 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949)
I20260807 02:26:50.182953 19978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c2f13c2d74a81bc933635fc5e1f64" candidate_uuid: "03a739030832483a9926c0ae183ae1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c"
I20260807 02:26:50.182966 19550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676c2f13c2d74a81bc933635fc5e1f64" candidate_uuid: "03a739030832483a9926c0ae183ae1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec319e391af4f5ea85980bd8f69b445"
I20260807 02:26:50.183089 19978 raft_consensus.cc:3037] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.183107 19550 raft_consensus.cc:3037] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.184237 19550 raft_consensus.cc:2454] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a739030832483a9926c0ae183ae1a7 in term 1.
I20260807 02:26:50.184326 19978 raft_consensus.cc:2454] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a739030832483a9926c0ae183ae1a7 in term 1.
I20260807 02:26:50.184561 19667 leader_election.cc:302] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a739030832483a9926c0ae183ae1a7, 4ec319e391af4f5ea85980bd8f69b445; no voters:
I20260807 02:26:50.184912 20093 raft_consensus.cc:2781] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:50.185165 20093 raft_consensus.cc:686] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 1 LEADER]: Becoming Leader. State: Replica: 03a739030832483a9926c0ae183ae1a7, State: Running, Role: LEADER
I20260807 02:26:50.185532 20093 consensus_queue.cc:237] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } }
I20260807 02:26:50.189544 19319 catalog_manager.cc:5495] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03a739030832483a9926c0ae183ae1a7 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "03a739030832483a9926c0ae183ae1a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } health_report { overall_health: UNKNOWN } } }
I20260807 02:26:50.210346 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.206557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.17.53.2:33889)
0807 02:26:50.206698 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.206702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.206717 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:26:50.206922 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.206924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.206983 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.207073 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:50.207362 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.207856 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.208733 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.209028 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.209111 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.209457 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.209878 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.209987 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.210178 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.210231 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:26:50.216346 19550 raft_consensus.cc:1261] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Refusing update from remote peer 03a739030832483a9926c0ae183ae1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.216346 19978 raft_consensus.cc:1261] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Refusing update from remote peer 03a739030832483a9926c0ae183ae1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.217195 20099 consensus_queue.cc:1035] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.217398 20093 consensus_queue.cc:1035] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.228667 20107 mvcc.cc:205] Tried to move back new op lower bound from 7315741123438125056 to 7315741123323404288. Current Snapshot: MvccSnapshot[applied={T|T < 7315741123438125056 or (T in {7315741123438125056})}]
I20260807 02:26:50.239974 19320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:50.240195 19320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:50.242795 19320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Sending DeleteTablet for 3 replicas of tablet 676c2f13c2d74a81bc933635fc5e1f64
I20260807 02:26:50.243592 19732 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:50008
I20260807 02:26:50.243641 19944 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:56252
I20260807 02:26:50.243877 19515 tablet_service.cc:1514] Processing DeleteTablet for tablet 676c2f13c2d74a81bc933635fc5e1f64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:58102
I20260807 02:26:50.244199 20122 tablet_replica.cc:330] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: stopping tablet replica
I20260807 02:26:50.244392 20123 tablet_replica.cc:330] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: stopping tablet replica
I20260807 02:26:50.244544 20122 raft_consensus.cc:2229] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:50.244814 20121 tablet_replica.cc:330] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: stopping tablet replica
I20260807 02:26:50.244908 20122 raft_consensus.cc:2258] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.245208 20121 raft_consensus.cc:2229] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:50.245398 19320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 02:26:50.245581 20123 raft_consensus.cc:2229] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 02:26:50.245817 19320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:50.245882 20123 raft_consensus.cc:2258] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.245985 20121 raft_consensus.cc:2258] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.247725 20121 ts_tablet_manager.cc:1905] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.247726 20123 ts_tablet_manager.cc:1905] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.248703 20122 ts_tablet_manager.cc:1905] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.251490 19515 tablet_service.cc:1467] Processing CreateTablet for tablet 3e684f1f2359413a8969ab038cd7ebee (DEFAULT_TABLE table=test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.251680 19944 tablet_service.cc:1467] Processing CreateTablet for tablet 3e684f1f2359413a8969ab038cd7ebee (DEFAULT_TABLE table=test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.252045 19515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e684f1f2359413a8969ab038cd7ebee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.252055 19944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e684f1f2359413a8969ab038cd7ebee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.254340 20081 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Bootstrap starting.
I20260807 02:26:50.254340 20083 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Bootstrap starting.
I20260807 02:26:50.254664 20123 ts_tablet_manager.cc:1918] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:26:50.254768 20123 log.cc:1196] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/676c2f13c2d74a81bc933635fc5e1f64
I20260807 02:26:50.255038 19732 tablet_service.cc:1467] Processing CreateTablet for tablet 3e684f1f2359413a8969ab038cd7ebee (DEFAULT_TABLE table=test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:50.255293 19732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e684f1f2359413a8969ab038cd7ebee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.255388 20081 tablet_bootstrap.cc:654] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.255388 20083 tablet_bootstrap.cc:654] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.255439 20123 ts_tablet_manager.cc:1939] T 676c2f13c2d74a81bc933635fc5e1f64 P 4ec319e391af4f5ea85980bd8f69b445: Deleting consensus metadata
I20260807 02:26:50.256414 20121 ts_tablet_manager.cc:1918] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:26:50.256527 20122 ts_tablet_manager.cc:1918] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:26:50.256668 20121 log.cc:1196] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/676c2f13c2d74a81bc933635fc5e1f64
I20260807 02:26:50.256755 19296 catalog_manager.cc:4841] TS 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949): tablet 676c2f13c2d74a81bc933635fc5e1f64 (table test-table [id=facecdc2d55f4e64856e96d1baad4abf]) successfully deleted
I20260807 02:26:50.256827 20122 log.cc:1196] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/676c2f13c2d74a81bc933635fc5e1f64
I20260807 02:26:50.257228 20121 ts_tablet_manager.cc:1939] T 676c2f13c2d74a81bc933635fc5e1f64 P 03a739030832483a9926c0ae183ae1a7: Deleting consensus metadata
I20260807 02:26:50.257589 20122 ts_tablet_manager.cc:1939] T 676c2f13c2d74a81bc933635fc5e1f64 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting consensus metadata
I20260807 02:26:50.257967 20081 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: No bootstrap required, opened a new log
I20260807 02:26:50.258061 20081 ts_tablet_manager.cc:1397] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 02:26:50.258108 20085 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Bootstrap starting.
I20260807 02:26:50.258953 20081 raft_consensus.cc:348] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.259097 20081 raft_consensus.cc:374] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.259140 20085 tablet_bootstrap.cc:654] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.259148 20081 raft_consensus.cc:729] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec319e391af4f5ea85980bd8f69b445, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.259295 20081 consensus_queue.cc:260] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.259605 20081 ts_tablet_manager.cc:1428] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 02:26:50.260030 19294 catalog_manager.cc:4841] TS 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889): tablet 676c2f13c2d74a81bc933635fc5e1f64 (table test-table [id=facecdc2d55f4e64856e96d1baad4abf]) successfully deleted
I20260807 02:26:50.260257 19297 catalog_manager.cc:4841] TS a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281): tablet 676c2f13c2d74a81bc933635fc5e1f64 (table test-table [id=facecdc2d55f4e64856e96d1baad4abf]) successfully deleted
I20260807 02:26:50.261447 20085 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: No bootstrap required, opened a new log
I20260807 02:26:50.261538 20085 ts_tablet_manager.cc:1397] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:26:50.262006 20085 raft_consensus.cc:348] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.262115 20085 raft_consensus.cc:374] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.262152 20085 raft_consensus.cc:729] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a739030832483a9926c0ae183ae1a7, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.262303 20085 consensus_queue.cc:260] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.262584 20085 ts_tablet_manager.cc:1428] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:50.277503 20083 tablet_bootstrap.cc:492] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: No bootstrap required, opened a new log
I20260807 02:26:50.277613 20083 ts_tablet_manager.cc:1397] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260807 02:26:50.278110 20083 raft_consensus.cc:348] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.278226 20083 raft_consensus.cc:374] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.278258 20083 raft_consensus.cc:729] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77ea3f8d14a48f7a3b30ffd5ac6a85c, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.278405 20083 consensus_queue.cc:260] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.278664 20083 ts_tablet_manager.cc:1428] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 02:26:50.309222 20048 tablet.cc:2369] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:50.310626 20097 raft_consensus.cc:482] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:50.310765 20097 raft_consensus.cc:504] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.312009 20097 leader_election.cc:288] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949), 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889)
I20260807 02:26:50.316556 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.312216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:33889 (local address 127.17.53.3:50415)
0807 02:26:50.312415 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.312430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.312520 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.312879 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.312882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.312901 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.313030 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.313039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.313967 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.313971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.315318 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.315329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.316188 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.316191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.316316 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.316335 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.316397 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.316446 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":8}
I20260807 02:26:50.316588 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.312436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:42817 (local address 127.17.53.1:41949)
0807 02:26:50.312565 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.312571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.313027 (+ 456us) server_negotiation.cc:407] Connection header received
0807 02:26:50.313202 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.313206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.313266 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.313345 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.313645 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.314252 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.315397 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.315807 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.315950 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.316005 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.316372 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.316461 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:26:50.316574 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.312232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41949 (local address 127.17.53.3:42817)
0807 02:26:50.312912 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.312926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.313043 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.313379 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.313382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.313402 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.313524 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.313530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.314374 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.314378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.315272 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.315281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.316074 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.316076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.316184 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.316240 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.316319 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.316404 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":397,"thread_start_us":77,"threads_started":1}
I20260807 02:26:50.317065 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.312340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50415 (local address 127.17.53.2:33889)
0807 02:26:50.312459 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.312464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.312479 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:26:50.312674 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.312681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.312752 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.312869 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.313152 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.313850 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.315470 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.316074 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.316182 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.316247 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.316873 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.316949 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:26:50.317066 19550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e684f1f2359413a8969ab038cd7ebee" candidate_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec319e391af4f5ea85980bd8f69b445" is_pre_election: true
I20260807 02:26:50.317226 19550 raft_consensus.cc:2454] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a77ea3f8d14a48f7a3b30ffd5ac6a85c in term 0.
W20260807 02:26:50.317597 19625 tablet.cc:2369] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:50.317562 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e684f1f2359413a8969ab038cd7ebee" candidate_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739030832483a9926c0ae183ae1a7" is_pre_election: true
I20260807 02:26:50.317798 19764 raft_consensus.cc:2454] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a77ea3f8d14a48f7a3b30ffd5ac6a85c in term 0.
I20260807 02:26:50.318167 19875 leader_election.cc:302] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec319e391af4f5ea85980bd8f69b445, a77ea3f8d14a48f7a3b30ffd5ac6a85c; no voters:
I20260807 02:26:50.318480 20097 raft_consensus.cc:2781] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:50.318543 20097 raft_consensus.cc:482] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:50.318588 20097 raft_consensus.cc:3037] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.319610 20097 raft_consensus.cc:504] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.319986 20097 leader_election.cc:288] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [CANDIDATE]: Term 1 election: Requested vote from peers 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949), 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889)
I20260807 02:26:50.320367 19550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e684f1f2359413a8969ab038cd7ebee" candidate_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec319e391af4f5ea85980bd8f69b445"
I20260807 02:26:50.320384 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e684f1f2359413a8969ab038cd7ebee" candidate_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739030832483a9926c0ae183ae1a7"
I20260807 02:26:50.320463 19550 raft_consensus.cc:3037] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.320489 19764 raft_consensus.cc:3037] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.321426 19550 raft_consensus.cc:2454] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a77ea3f8d14a48f7a3b30ffd5ac6a85c in term 1.
I20260807 02:26:50.321722 19764 raft_consensus.cc:2454] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a77ea3f8d14a48f7a3b30ffd5ac6a85c in term 1.
I20260807 02:26:50.321745 19875 leader_election.cc:302] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec319e391af4f5ea85980bd8f69b445, a77ea3f8d14a48f7a3b30ffd5ac6a85c; no voters:
I20260807 02:26:50.321911 20097 raft_consensus.cc:2781] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:50.322177 20097 raft_consensus.cc:686] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 LEADER]: Becoming Leader. State: Replica: a77ea3f8d14a48f7a3b30ffd5ac6a85c, State: Running, Role: LEADER
I20260807 02:26:50.322444 20097 consensus_queue.cc:237] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
W20260807 02:26:50.324838 19837 tablet.cc:2369] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:50.324752 19320 catalog_manager.cc:5495] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c reported cstate change: term changed from 0 to 1, leader changed from <none> to a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3). New cstate: current_term: 1 leader_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } health_report { overall_health: HEALTHY } } }
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069609, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:50.376549 20149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.366710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:43151 (local address 127.0.0.1:51194)
0807 02:26:50.367064 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.367082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.367217 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.367542 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.367545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.367923 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:50.368016 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.368022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.369107 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.369110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.369837 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.369849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.370451 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.370454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.370522 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.371243 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:50.371256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:50.372913 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:50.374793 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.374802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.374807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.375072 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.375399 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:50.375402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:50.375403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:50.375488 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:50.376016 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:50.376022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:50.376133 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.376267 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.376387 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":165,"threads_started":1}
I20260807 02:26:50.376741 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.366796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.17.53.62:43151)
0807 02:26:50.366955 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.366960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.367183 (+ 223us) server_negotiation.cc:407] Connection header received
0807 02:26:50.367324 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.367328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.367383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.367492 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.368154 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.368985 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.369968 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.370312 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.370433 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.373055 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.373060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.373062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.373089 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.374652 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.375192 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.375195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.375197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.375294 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.375591 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.375595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.375597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.375891 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.376003 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.376364 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.376531 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 02:26:50.381596 19320 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:51194:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 02:26:50.382438 19320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:50.388167 19515 tablet_service.cc:1467] Processing CreateTablet for tablet 10a7e4946b0047098e074946c9c8d902 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4cf3f38f42744e196d097d0173f1be0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:50.388370 19732 tablet_service.cc:1467] Processing CreateTablet for tablet 10a7e4946b0047098e074946c9c8d902 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4cf3f38f42744e196d097d0173f1be0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:50.388630 19515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10a7e4946b0047098e074946c9c8d902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.388826 19732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10a7e4946b0047098e074946c9c8d902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.391680 19944 tablet_service.cc:1467] Processing CreateTablet for tablet 10a7e4946b0047098e074946c9c8d902 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4cf3f38f42744e196d097d0173f1be0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:50.391867 20081 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: Bootstrap starting.
I20260807 02:26:50.392059 19944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10a7e4946b0047098e074946c9c8d902. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:50.392589 20085 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: Bootstrap starting.
I20260807 02:26:50.392994 20081 tablet_bootstrap.cc:654] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.393621 20085 tablet_bootstrap.cc:654] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.394786 20081 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: No bootstrap required, opened a new log
I20260807 02:26:50.394866 20081 ts_tablet_manager.cc:1397] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:26:50.395326 20081 raft_consensus.cc:348] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.395439 20083 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Bootstrap starting.
I20260807 02:26:50.395455 20081 raft_consensus.cc:374] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.395498 20081 raft_consensus.cc:729] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec319e391af4f5ea85980bd8f69b445, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.395627 20081 consensus_queue.cc:260] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.395926 20081 ts_tablet_manager.cc:1428] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:50.396179 20095 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.396358 20095 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.396402 20083 tablet_bootstrap.cc:654] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:50.397714 20083 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: No bootstrap required, opened a new log
I20260807 02:26:50.397737 20085 tablet_bootstrap.cc:492] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: No bootstrap required, opened a new log
I20260807 02:26:50.397800 20083 ts_tablet_manager.cc:1397] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 02:26:50.397823 20085 ts_tablet_manager.cc:1397] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 02:26:50.398274 20085 raft_consensus.cc:348] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.398416 20085 raft_consensus.cc:374] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.398459 20085 raft_consensus.cc:729] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a739030832483a9926c0ae183ae1a7, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.398531 20083 raft_consensus.cc:348] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.398644 20083 raft_consensus.cc:374] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:50.398681 20083 raft_consensus.cc:729] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77ea3f8d14a48f7a3b30ffd5ac6a85c, State: Initialized, Role: FOLLOWER
I20260807 02:26:50.398593 20085 consensus_queue.cc:260] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.398829 20083 consensus_queue.cc:260] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.398942 20113 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.399093 20113 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.399149 20083 ts_tablet_manager.cc:1428] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 02:26:50.398959 20085 ts_tablet_manager.cc:1428] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:26:50.399112 20097 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.399266 20097 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.403594 20095 raft_consensus.cc:482] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:50.403716 20095 raft_consensus.cc:504] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.405015 20095 leader_election.cc:288] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889), a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281)
I20260807 02:26:50.410810 20155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.405359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33281 (local address 127.17.53.1:35697)
0807 02:26:50.406368 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.406377 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.406507 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.406802 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.406804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.406822 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.406948 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.406959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.407949 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.407952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.409475 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.409486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.410339 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.410343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.410450 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.410490 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.410570 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.410648 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":112,"thread_start_us":157,"threads_started":1}
I20260807 02:26:50.410822 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.405787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35697 (local address 127.17.53.3:33281)
0807 02:26:50.405900 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.405904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.406453 (+ 549us) server_negotiation.cc:407] Connection header received
0807 02:26:50.406603 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.406606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.406646 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.406776 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.407082 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.407839 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.409614 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.410213 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.410322 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.410390 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.410597 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.410672 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:26:50.411204 19978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10a7e4946b0047098e074946c9c8d902" candidate_uuid: "4ec319e391af4f5ea85980bd8f69b445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" is_pre_election: true
I20260807 02:26:50.411352 19978 raft_consensus.cc:2454] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ec319e391af4f5ea85980bd8f69b445 in term 0.
I20260807 02:26:50.411438 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.405781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:46665 (local address 127.17.53.2:33889)
0807 02:26:50.405904 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.405907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.406157 (+ 250us) server_negotiation.cc:407] Connection header received
0807 02:26:50.406310 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.406313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.406359 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.406439 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:50.406800 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.407712 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.409978 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.410441 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.410524 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.410598 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.411244 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.411318 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:26:50.411696 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.405347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:33889 (local address 127.17.53.1:46665)
0807 02:26:50.406052 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.406067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.406214 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.406473 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.406478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.406496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:50.406634 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.406661 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.407853 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.407855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.409835 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.409848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.410701 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.410704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.410797 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.410826 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.411226 (+ 400us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.411298 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":430,"thread_start_us":157,"threads_started":1}
I20260807 02:26:50.411732 19450 leader_election.cc:302] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec319e391af4f5ea85980bd8f69b445, a77ea3f8d14a48f7a3b30ffd5ac6a85c; no voters:
I20260807 02:26:50.411993 20095 raft_consensus.cc:2781] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:50.412066 20095 raft_consensus.cc:482] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:50.412117 20095 raft_consensus.cc:3037] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.412354 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10a7e4946b0047098e074946c9c8d902" candidate_uuid: "4ec319e391af4f5ea85980bd8f69b445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739030832483a9926c0ae183ae1a7" is_pre_election: true
I20260807 02:26:50.412485 19764 raft_consensus.cc:2454] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ec319e391af4f5ea85980bd8f69b445 in term 0.
I20260807 02:26:50.413347 20095 raft_consensus.cc:504] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.413722 20095 leader_election.cc:288] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [CANDIDATE]: Term 1 election: Requested vote from peers 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889), a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281)
I20260807 02:26:50.414041 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10a7e4946b0047098e074946c9c8d902" candidate_uuid: "4ec319e391af4f5ea85980bd8f69b445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a739030832483a9926c0ae183ae1a7"
I20260807 02:26:50.414161 19764 raft_consensus.cc:3037] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.414143 19978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10a7e4946b0047098e074946c9c8d902" candidate_uuid: "4ec319e391af4f5ea85980bd8f69b445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c"
I20260807 02:26:50.414247 19978 raft_consensus.cc:3037] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:50.415011 19764 raft_consensus.cc:2454] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec319e391af4f5ea85980bd8f69b445 in term 1.
I20260807 02:26:50.415359 19978 raft_consensus.cc:2454] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec319e391af4f5ea85980bd8f69b445 in term 1.
I20260807 02:26:50.415697 19450 leader_election.cc:302] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03a739030832483a9926c0ae183ae1a7, 4ec319e391af4f5ea85980bd8f69b445, a77ea3f8d14a48f7a3b30ffd5ac6a85c; no voters:
I20260807 02:26:50.415951 20095 raft_consensus.cc:2781] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:50.416204 20095 raft_consensus.cc:686] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec319e391af4f5ea85980bd8f69b445, State: Running, Role: LEADER
I20260807 02:26:50.416476 20095 consensus_queue.cc:237] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } }
I20260807 02:26:50.417847 20160 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: TxnStatusTablet state changed. Reason: New leader 4ec319e391af4f5ea85980bd8f69b445. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.417971 20160 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:50.418268 20163 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 02:26:50.418452 20163 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 02:26:50.419112 19320 catalog_manager.cc:5495] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } health_report { overall_health: UNKNOWN } } }
I20260807 02:26:50.439221 20149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.434559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41949 (local address 127.0.0.1:58104)
0807 02:26:50.434727 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.434750 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.434862 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.435262 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.435264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.435273 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:50.435374 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.435379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.436453 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.436456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.437254 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.437266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.437978 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.437981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.438062 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.438118 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.438927 (+ 809us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.438943 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.439012 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.439091 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 02:26:50.439250 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.434656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.17.53.1:41949)
0807 02:26:50.434799 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.434802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.434817 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:26:50.435049 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.435053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.435120 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.435251 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:50.435488 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.436319 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.437405 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.437808 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.437942 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.438291 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.438820 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.438947 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.439051 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.439106 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:26:50.442723 19978 raft_consensus.cc:1261] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Refusing update from remote peer 4ec319e391af4f5ea85980bd8f69b445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.442855 19764 raft_consensus.cc:1261] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Refusing update from remote peer 4ec319e391af4f5ea85980bd8f69b445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.443297 20160 consensus_queue.cc:1035] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.443481 20095 consensus_queue.cc:1035] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.445111 20097 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: TxnStatusTablet state changed. Reason: New leader 4ec319e391af4f5ea85980bd8f69b445. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.445256 20097 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.445436 20113 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: TxnStatusTablet state changed. Reason: New leader 4ec319e391af4f5ea85980bd8f69b445. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.445554 20113 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.447274 20095 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.447381 20095 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:50.447558 20097 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.447650 20097 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P a77ea3f8d14a48f7a3b30ffd5ac6a85c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.448988 20108 mvcc.cc:205] Tried to move back new op lower bound from 7315741124367712256 to 7315741124268642304. Current Snapshot: MvccSnapshot[applied={T|T < 7315741124367712256 or (T in {7315741124367712256})}]
I20260807 02:26:50.452368 20160 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.452481 20160 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 4ec319e391af4f5ea85980bd8f69b445: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:50.453157 20105 mvcc.cc:205] Tried to move back new op lower bound from 7315741124367712256 to 7315741124268642304. Current Snapshot: MvccSnapshot[applied={T|T < 7315741124367712256}]
I20260807 02:26:50.453358 20113 tablet_replica.cc:439] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ec319e391af4f5ea85980bd8f69b445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 } } peers { permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 } } peers { permanent_uuid: "a77ea3f8d14a48f7a3b30ffd5ac6a85c" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33281 } } }
I20260807 02:26:50.453462 20113 tablet_replica.cc:442] T 10a7e4946b0047098e074946c9c8d902 P 03a739030832483a9926c0ae183ae1a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:50.458978 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.454460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.17.53.3:33281)
0807 02:26:50.454585 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.454590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.454607 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:26:50.454802 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.454806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.454885 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.454994 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:50.455433 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.456217 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.457252 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.457595 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.457702 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.457984 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.458516 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.458634 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.458798 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.458852 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 02:26:50.470388 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.466740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34529 (local address 127.17.53.1:41949)
0807 02:26:50.466863 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.466867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.466881 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:26:50.466978 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.466981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.467027 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.467098 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:50.467359 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.468128 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.469083 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.469436 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.469539 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.469816 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.469929 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.470055 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.470210 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.470267 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 02:26:50.470383 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.466630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41949 (local address 127.17.53.3:34529)
0807 02:26:50.466781 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.466799 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.466886 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.467124 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.467127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.467134 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:50.467243 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.467248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.468289 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.468292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.468961 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.468970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.469551 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.469554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.469623 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.469674 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.470080 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.470088 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.470180 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.470248 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 02:26:50.478385 19550 raft_consensus.cc:1261] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Refusing update from remote peer a77ea3f8d14a48f7a3b30ffd5ac6a85c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.478494 19764 raft_consensus.cc:1261] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Refusing update from remote peer a77ea3f8d14a48f7a3b30ffd5ac6a85c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:50.478897 20135 consensus_queue.cc:1035] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec319e391af4f5ea85980bd8f69b445" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.479097 20097 consensus_queue.cc:1035] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a739030832483a9926c0ae183ae1a7" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 33889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:50.502910 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.498471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33281 (local address 127.17.53.1:57075)
0807 02:26:50.498830 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:50.498843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:50.498939 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:50.499154 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:50.499156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:50.499161 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:50.499342 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.499349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.500374 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.500378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.501387 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:50.501399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.502056 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.502060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:50.502160 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.502227 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.502575 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.502581 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:26:50.502650 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:26:50.502737 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":120,"threads_started":1}
I20260807 02:26:50.502954 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.498582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:57075 (local address 127.17.53.3:33281)
0807 02:26:50.498795 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.498800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.498926 (+ 126us) server_negotiation.cc:407] Connection header received
0807 02:26:50.499016 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.499019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.499069 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.499146 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:50.499469 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.500244 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.501542 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.501926 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.502024 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.502366 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:50.502472 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:50.502555 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.502746 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.502812 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 02:26:50.533631 19319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:50.533866 19319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:50.536687 19319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ea9d8e40a84a18a2dfd378582dbfe1: Sending DeleteTablet for 3 replicas of tablet 3e684f1f2359413a8969ab038cd7ebee
I20260807 02:26:50.537608 19944 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e684f1f2359413a8969ab038cd7ebee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:56252
I20260807 02:26:50.537657 19515 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e684f1f2359413a8969ab038cd7ebee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:58102
I20260807 02:26:50.537830 20122 tablet_replica.cc:330] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: stopping tablet replica
I20260807 02:26:50.537859 20123 tablet_replica.cc:330] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: stopping tablet replica
I20260807 02:26:50.537961 20122 raft_consensus.cc:2229] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:50.537971 20123 raft_consensus.cc:2229] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:50.538097 20123 raft_consensus.cc:2258] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.538297 20122 raft_consensus.cc:2258] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.539193 20122 ts_tablet_manager.cc:1905] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.539193 20123 ts_tablet_manager.cc:1905] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.541509 19732 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e684f1f2359413a8969ab038cd7ebee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:50008
I20260807 02:26:50.541738 20121 tablet_replica.cc:330] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: stopping tablet replica
I20260807 02:26:50.541913 20121 raft_consensus.cc:2229] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:50.542052 20121 raft_consensus.cc:2258] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:50.543013 20121 ts_tablet_manager.cc:1905] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:50.544523 20123 ts_tablet_manager.cc:1918] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:50.544597 20123 log.cc:1196] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/3e684f1f2359413a8969ab038cd7ebee
I20260807 02:26:50.544899 20123 ts_tablet_manager.cc:1939] T 3e684f1f2359413a8969ab038cd7ebee P 4ec319e391af4f5ea85980bd8f69b445: Deleting consensus metadata
I20260807 02:26:50.545915 19296 catalog_manager.cc:4841] TS 4ec319e391af4f5ea85980bd8f69b445 (127.17.53.1:41949): tablet 3e684f1f2359413a8969ab038cd7ebee (table test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]) successfully deleted
I20260807 02:26:50.546209 20121 ts_tablet_manager.cc:1918] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:50.546304 20121 log.cc:1196] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/3e684f1f2359413a8969ab038cd7ebee
I20260807 02:26:50.546609 20121 ts_tablet_manager.cc:1939] T 3e684f1f2359413a8969ab038cd7ebee P 03a739030832483a9926c0ae183ae1a7: Deleting consensus metadata
I20260807 02:26:50.547358 19294 catalog_manager.cc:4841] TS 03a739030832483a9926c0ae183ae1a7 (127.17.53.2:33889): tablet 3e684f1f2359413a8969ab038cd7ebee (table test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]) successfully deleted
I20260807 02:26:50.548501 20122 ts_tablet_manager.cc:1918] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:50.548601 20122 log.cc:1196] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/3e684f1f2359413a8969ab038cd7ebee
I20260807 02:26:50.548902 20122 ts_tablet_manager.cc:1939] T 3e684f1f2359413a8969ab038cd7ebee P a77ea3f8d14a48f7a3b30ffd5ac6a85c: Deleting consensus metadata
I20260807 02:26:50.549650 19297 catalog_manager.cc:4841] TS a77ea3f8d14a48f7a3b30ffd5ac6a85c (127.17.53.3:33281): tablet 3e684f1f2359413a8969ab038cd7ebee (table test-table [id=fd3e662da6994ddbaa87c1571fabdbcc]) successfully deleted
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[19250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:26:50.562765 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.546958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.17.53.1:41949)
0807 02:26:50.547073 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.547078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.549824 (+ 2746us) server_negotiation.cc:407] Connection header received
0807 02:26:50.550042 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.550046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.550110 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.550218 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.551179 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.552072 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.553346 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.553736 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.553856 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.558048 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.558059 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.558076 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.558107 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.560312 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.560912 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.560919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.560922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.561051 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.561439 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.561444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.561446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.561837 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.562064 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.562401 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.562559 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 02:26:50.563308 19616 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58112
I20260807 02:26:50.579660 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:50.565468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.17.53.62:43151)
0807 02:26:50.565617 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:50.565622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:50.569066 (+ 3444us) server_negotiation.cc:407] Connection header received
0807 02:26:50.569432 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:50.569436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:50.569498 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:50.569557 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:50.570593 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.571431 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.572690 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:50.573140 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:50.573241 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:50.575121 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:50.575126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:50.575129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:50.575161 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:50.577020 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.577813 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.577818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.577820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.577930 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:50.578324 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:50.578328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:50.578330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:50.578680 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:50.578795 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:50.579336 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 02:26:50.579474 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 02:26:50.581004 19319 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204
I20260807 02:26:50.583379 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19424
I20260807 02:26:50.599028 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19637
I20260807 02:26:50.618011 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19847
I20260807 02:26:50.633431 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19271
2026-08-07T02:26:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2962 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[20197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[20197](info): set up 2 sockets
Aug 07 02:26:50 dist-test-slave-g170 krb5kdc[20197](info): commencing operation
krb5kdc: starting...
W20260807 02:26:51.878508 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:26:51 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:26:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:26:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:26:52.014460 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38273
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:38273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:52.128506 20220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:52.128840 20220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:52.128957 20220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:52.133016 20220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:26:52.133101 20220 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 02:26:52.133132 20220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:52.133178 20220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:52.133213 20220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:26:52.133255 20220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:26:52.138463 20220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:38273
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:52.139532 20220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:52.140588 20220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:52.149374 20227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:52.149485 20220 server_base.cc:1034] running on GCE node
W20260807 02:26:52.149408 20230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.149386 20228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:52.150249 20220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:52.151324 20220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:52.152489 20220 hybrid_clock.cc:648] HybridClock initialized: now 1786069612152459 us; error 43 us; skew 500 ppm
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:52.155339 20220 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:26:52.156244 20220 webserver.cc:460] Webserver started at http://127.17.53.62:46231/ using document root <none> and password file <none>
I20260807 02:26:52.156888 20220 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:52.156981 20220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:52.157240 20220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:52.159277 20220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "8290323db0c74aaa9bd401737a71b90e"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "71c9be6cb93c5c988fc7ee14edec6dca"
server_key_iv: "bb686773605d34e4bd21bdcb2d28123c"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.159837 20220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "8290323db0c74aaa9bd401737a71b90e"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "71c9be6cb93c5c988fc7ee14edec6dca"
server_key_iv: "bb686773605d34e4bd21bdcb2d28123c"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.163517 20220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 02:26:52.166036 20241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.167256 20220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:26:52.167410 20220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "8290323db0c74aaa9bd401737a71b90e"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "71c9be6cb93c5c988fc7ee14edec6dca"
server_key_iv: "bb686773605d34e4bd21bdcb2d28123c"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.167521 20220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:52.187881 20220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:52.190935 20220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:52.191143 20220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:52.199674 20220 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:38273
I20260807 02:26:52.199699 20341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:38273 every 8 connection(s)
I20260807 02:26:52.200624 20220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:26:52.201779 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 20220
I20260807 02:26:52.201901 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:26:52.202198 17620 external_mini_cluster.cc:1428] Setting key 5be39446931676b2a5edc43ec7c647e0
I20260807 02:26:52.204016 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:52.209867 20342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Bootstrap starting.
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:52.212440 20342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:52.213665 20342 log.cc:824] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:52.215996 20342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: No bootstrap required, opened a new log
I20260807 02:26:52.217031 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.203664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.17.53.62:38273)
0807 02:26:52.204186 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.204203 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.204250 (+ 47us) server_negotiation.cc:407] Connection header received
0807 02:26:52.204969 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.204988 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.205325 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.205666 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.206500 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.207495 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.208364 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.208954 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.209196 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.211974 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.211979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.211992 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.212018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.213847 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.214335 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.214345 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.214353 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.214504 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.214862 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.214865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.214867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.215275 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.215452 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.215803 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.216238 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":156,"threads_started":1}
I20260807 02:26:52.219086 20342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } }
I20260807 02:26:52.219329 20342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:52.219424 20342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8290323db0c74aaa9bd401737a71b90e, State: Initialized, Role: FOLLOWER
I20260807 02:26:52.219877 20342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } }
I20260807 02:26:52.220045 20342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:26:52.220130 20342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:26:52.220250 20342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:52.221226 20342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } }
I20260807 02:26:52.221561 20342 leader_election.cc:302] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8290323db0c74aaa9bd401737a71b90e; no voters:
I20260807 02:26:52.221922 20342 leader_election.cc:288] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:26:52.221995 20349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:52.222205 20349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [term 1 LEADER]: Becoming Leader. State: Replica: 8290323db0c74aaa9bd401737a71b90e, State: Running, Role: LEADER
I20260807 02:26:52.222503 20349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } }
I20260807 02:26:52.223230 20342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:26:52.223762 20351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8290323db0c74aaa9bd401737a71b90e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } } }
I20260807 02:26:52.223814 20354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8290323db0c74aaa9bd401737a71b90e. Latest consensus state: current_term: 1 leader_uuid: "8290323db0c74aaa9bd401737a71b90e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8290323db0c74aaa9bd401737a71b90e" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38273 } } }
I20260807 02:26:52.224011 20354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:52.224004 20351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:52.224694 20364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:26:52.228058 20364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:26:52.234535 20364 catalog_manager.cc:1347] Generated new cluster ID: 66c47a3046c34c17ab5f699dc84d3f22
I20260807 02:26:52.234628 20364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:26:52.256247 20364 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:26:52.257393 20364 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:26:52.261495 20364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Generated new TSK 0
I20260807 02:26:52.262224 20364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:26:52.303328 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38273
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:52.421144 20389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:52.421409 20389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:52.421489 20389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:52.425765 20389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:52.425829 20389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:52.425870 20389 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 02:26:52.425907 20389 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:52.425972 20389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:26:52.431550 20389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:52.432639 20389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:52.433727 20389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:52.443511 20396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.443615 20398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.443506 20400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:52.443773 20389 server_base.cc:1034] running on GCE node
I20260807 02:26:52.444262 20389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:52.445052 20389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:52.446269 20389 hybrid_clock.cc:648] HybridClock initialized: now 1786069612446244 us; error 41 us; skew 500 ppm
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:52.449410 20389 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:26:52.450385 20389 webserver.cc:460] Webserver started at http://127.17.53.1:38007/ using document root <none> and password file <none>
I20260807 02:26:52.450940 20389 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:52.451001 20389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:52.451174 20389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:52.453334 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "c96d4206efef4c98ab336eb89fcbe607"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "cb5b867c9af90a44d079b3355f03ab9e"
server_key_iv: "2dd5157c97d3b35b0ebf5138674820fd"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.454118 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "c96d4206efef4c98ab336eb89fcbe607"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "cb5b867c9af90a44d079b3355f03ab9e"
server_key_iv: "2dd5157c97d3b35b0ebf5138674820fd"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.458338 20389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:26:52.460942 20410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.462354 20389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:52.462466 20389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "c96d4206efef4c98ab336eb89fcbe607"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "cb5b867c9af90a44d079b3355f03ab9e"
server_key_iv: "2dd5157c97d3b35b0ebf5138674820fd"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.462553 20389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:52.471738 20389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:52.474880 20389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:52.475033 20389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:52.476464 20389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:52.476513 20389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.476558 20389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:52.476612 20389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:52.488651 20389 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45239
I20260807 02:26:52.488747 20586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45239 every 8 connection(s)
I20260807 02:26:52.489611 20389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:26:52.490522 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 20389
I20260807 02:26:52.490667 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:26:52.491030 17620 external_mini_cluster.cc:1428] Setting key e171ac56b0d3206efa53991f752981b4
I20260807 02:26:52.491787 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.477270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:39453 (local address 127.17.53.62:38273)
0807 02:26:52.477463 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.477470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.478374 (+ 904us) server_negotiation.cc:407] Connection header received
0807 02:26:52.479695 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.479701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.479770 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.479839 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.481735 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.482303 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.483687 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.484109 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.484251 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.487270 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.487274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.487277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.487307 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.489124 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.489697 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.489703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.489705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.489809 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.490483 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.490487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.490489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.490781 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.490907 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.491436 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.491580 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 02:26:52.492714 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.477664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.1:39453)
0807 02:26:52.478173 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.478231 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.479423 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.480311 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.480325 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.480855 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.481574 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.481596 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.482455 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.482459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.483548 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.483560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.484347 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.484355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.484573 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.485228 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.485250 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.487083 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.489262 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.489270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.489285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.489568 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.489929 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.489934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.489936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.490143 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.490902 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.490907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.491167 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.491341 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.491740 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260807 02:26:52.494004 20587 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38273
I20260807 02:26:52.494177 20587 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:52.494668 20587 heartbeater.cc:511] Master 127.17.53.62:38273 requested a full tablet report, sending...
I20260807 02:26:52.496187 20265 ts_manager.cc:194] Registered new tserver with Master: c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239)
I20260807 02:26:52.497498 20265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:39453
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:26:52.504184 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.497051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49199 (local address 127.17.53.62:38273)
0807 02:26:52.497187 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.497194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.497211 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:26:52.497251 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.497255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.497295 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.497411 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.497969 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.498661 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.499503 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.499746 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.499860 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.501050 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.501054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.501057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.501080 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.502542 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.502981 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.502985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.502987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.503067 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.503358 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.503362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.503364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.503535 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.503653 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.503906 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.504017 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 02:26:52.504194 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.496665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.1:49199)
0807 02:26:52.497037 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.497053 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.497162 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.497418 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.497421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.497665 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.497833 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.497845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.498800 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.498803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.499377 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.499385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.499865 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.499869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.499965 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.500516 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.500528 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.500916 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.502666 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.502672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.502674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.502881 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.503170 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.503174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.503176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.503265 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.503635 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.503639 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.503742 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.503851 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.503993 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:26:52.525139 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38273
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:52.642900 20603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:52.643139 20603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:52.643184 20603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:52.647035 20603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:52.647083 20603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:52.647121 20603 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 02:26:52.647135 20603 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:52.647166 20603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:26:52.652261 20603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:52.653214 20603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:52.654162 20603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:52.663959 20609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.663928 20611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.663920 20613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:52.664227 20603 server_base.cc:1034] running on GCE node
I20260807 02:26:52.664613 20603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:52.665328 20603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:52.666534 20603 hybrid_clock.cc:648] HybridClock initialized: now 1786069612666503 us; error 51 us; skew 500 ppm
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:52.669514 20603 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:26:52.670521 20603 webserver.cc:460] Webserver started at http://127.17.53.2:40783/ using document root <none> and password file <none>
I20260807 02:26:52.671195 20603 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:52.671259 20603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:52.671424 20603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:52.673658 20603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "22fd916b80944e1db828d3d5c6e65536"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "a48386ba07253cf8fbccc12a39959acc"
server_key_iv: "4a4a9b3f9d1da6d3c823b7cfd37f3ba6"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.674185 20603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "22fd916b80944e1db828d3d5c6e65536"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "a48386ba07253cf8fbccc12a39959acc"
server_key_iv: "4a4a9b3f9d1da6d3c823b7cfd37f3ba6"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.678014 20603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 02:26:52.680483 20629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.681891 20603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:26:52.682000 20603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "22fd916b80944e1db828d3d5c6e65536"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "a48386ba07253cf8fbccc12a39959acc"
server_key_iv: "4a4a9b3f9d1da6d3c823b7cfd37f3ba6"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.682091 20603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:52.714984 20603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:52.718299 20603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:52.718482 20603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:52.719985 20603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:52.720041 20603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.720109 20603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:52.720144 20603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:52.732340 20603 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39559
I20260807 02:26:52.732928 20800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39559 every 8 connection(s)
I20260807 02:26:52.733433 20603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:26:52.735617 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.720924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:39471 (local address 127.17.53.62:38273)
0807 02:26:52.721106 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.721112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.722037 (+ 925us) server_negotiation.cc:407] Connection header received
0807 02:26:52.723452 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.723455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.723505 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.723583 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.725398 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.726195 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.728160 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.728399 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.728529 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.731726 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.731730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.731733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.731765 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.733470 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.734099 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.734104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.734105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.734205 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.734558 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.734561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.734562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.734771 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.734866 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.735385 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.735495 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 02:26:52.736447 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.721321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.2:39471)
0807 02:26:52.721836 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.721899 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.723151 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.723855 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.723868 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.724459 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.725215 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.725242 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.726353 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.726359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.727998 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.728012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.728606 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.728614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.728898 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.729610 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.729631 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.731537 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.733599 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.733605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.733620 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.733936 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.734328 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.734331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.734332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.734463 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.734903 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.734908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.735150 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.735313 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.735578 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260807 02:26:52.737802 20801 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38273
I20260807 02:26:52.738001 20801 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:52.738564 20801 heartbeater.cc:511] Master 127.17.53.62:38273 requested a full tablet report, sending...
I20260807 02:26:52.739849 20268 ts_manager.cc:194] Registered new tserver with Master: 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:52.740687 20268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:39471
I20260807 02:26:52.743327 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 20603
I20260807 02:26:52.743453 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:26:52.743721 17620 external_mini_cluster.cc:1428] Setting key 8ea9ac902d0f16d2d1e6eb0013bfb0e6
I20260807 02:26:52.748546 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.740356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:45019 (local address 127.17.53.62:38273)
0807 02:26:52.740508 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.740515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.740812 (+ 297us) server_negotiation.cc:407] Connection header received
0807 02:26:52.741026 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.741032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.741082 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.741194 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.741680 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.742192 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.743047 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.743277 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.743399 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.744721 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.744724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.744726 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.744748 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.746651 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.747104 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.747111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.747115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.747213 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.747495 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.747501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.747505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.747845 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.747975 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.748268 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.748379 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:26:52.748565 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.740235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.2:45019)
0807 02:26:52.740686 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.740708 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.740907 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.741213 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.741216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.741429 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.741544 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.741550 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.742332 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.742336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.742913 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.742921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.743406 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.743410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.743513 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.744141 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.744155 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.744577 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.746784 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.746787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.746788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.747003 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.747312 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.747314 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.747315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.747400 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.747967 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.747970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.748091 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.748208 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.748372 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":141,"threads_started":1}
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:26:52.776541 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38273
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:52.895434 20813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:52.895685 20813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:52.895732 20813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:52.899787 20813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:52.899848 20813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:52.899887 20813 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 02:26:52.899901 20813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:26:52.899932 20813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:26:52.905104 20813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:38273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:52.906061 20813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:52.907100 20813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:52.916744 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.916750 20822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:52.916750 20824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:52.917058 20813 server_base.cc:1034] running on GCE node
I20260807 02:26:52.917516 20813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:52.918210 20813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:52.919411 20813 hybrid_clock.cc:648] HybridClock initialized: now 1786069612919394 us; error 27 us; skew 500 ppm
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:52.922266 20813 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:26:52.923195 20813 webserver.cc:460] Webserver started at http://127.17.53.3:42373/ using document root <none> and password file <none>
I20260807 02:26:52.923766 20813 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:52.923837 20813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:52.924105 20813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:52.926292 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "4cf5f598d2c84f868e5a7a3b61354d25"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "c6b56b708c2657f03ec8b3f97a401be1"
server_key_iv: "ecaee60358c6e8e6c0d49f63637244b2"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.926800 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "4cf5f598d2c84f868e5a7a3b61354d25"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "c6b56b708c2657f03ec8b3f97a401be1"
server_key_iv: "ecaee60358c6e8e6c0d49f63637244b2"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.930606 20813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 02:26:52.933233 20833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.934460 20813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:26:52.934561 20813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "4cf5f598d2c84f868e5a7a3b61354d25"
format_stamp: "Formatted at 2026-08-07 02:26:52 on dist-test-slave-g170"
server_key: "c6b56b708c2657f03ec8b3f97a401be1"
server_key_iv: "ecaee60358c6e8e6c0d49f63637244b2"
server_key_version: "encryptionkey@0"
I20260807 02:26:52.934643 20813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:52.946877 20813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:52.949738 20813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:52.949901 20813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:52.951229 20813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:52.951278 20813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:52.951344 20813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:52.951376 20813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:52.962787 20813 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38833
I20260807 02:26:52.963029 21006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38833 every 8 connection(s)
I20260807 02:26:52.963757 20813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:26:52.964121 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 20813
I20260807 02:26:52.964255 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:26:52.964673 17620 external_mini_cluster.cc:1428] Setting key ec9f415aa60c7dda14e299d3506a31cb
I20260807 02:26:52.966190 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.952127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:52737 (local address 127.17.53.62:38273)
0807 02:26:52.952263 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.952268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.953394 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 02:26:52.954573 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.954578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.954648 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.954760 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.955933 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.956842 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.958030 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.958409 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.958533 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.961711 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.961717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.961720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.961753 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.963518 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.964201 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.964206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.964208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.964318 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.964828 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.964832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.964834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.965078 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.965225 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.965892 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.966018 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:26:52.967145 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.952671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.3:52737)
0807 02:26:52.953229 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.953273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.954321 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.954943 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.954949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.955288 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.955766 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.955780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.956997 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.957002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.957883 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.957894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.958631 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.958638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.958882 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.959624 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.959647 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.961511 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.963688 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.963695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.963716 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.964075 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.964456 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.964459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.964461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.964646 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.965297 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.965304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.965618 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.965845 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.966176 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":132,"threads_started":1}
I20260807 02:26:52.968545 21007 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38273
I20260807 02:26:52.968822 21007 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:52.969606 21007 heartbeater.cc:511] Master 127.17.53.62:38273 requested a full tablet report, sending...
I20260807 02:26:52.970777 20265 ts_manager.cc:194] Registered new tserver with Master: 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833)
I20260807 02:26:52.971547 20265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:52737
I20260807 02:26:52.978029 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.970998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:38669 (local address 127.17.53.62:38273)
0807 02:26:52.971125 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.971129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.971404 (+ 275us) server_negotiation.cc:407] Connection header received
0807 02:26:52.971523 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.971526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.971594 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:52.971680 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:52.972173 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.972674 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.973546 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.973779 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.973918 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.975073 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:52.975077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:52.975080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:52.975104 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:52.976348 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.976806 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.976811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.976816 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.976902 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:52.977199 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:52.977203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:52.977204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:52.977378 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:52.977487 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:52.977769 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 02:26:52.977879 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:26:52.978034 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.970918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.17.53.3:38669)
0807 02:26:52.971312 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:52.971326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:52.971445 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:52.971755 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:52.971759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:52.971938 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:52.972029 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.972035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.972837 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.972842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.973415 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:52.973423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:52.973924 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:52.973927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:52.974034 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:52.974557 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:52.974569 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:52.974932 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:52.976464 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.976469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.976471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.976679 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.977000 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:52.977003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:52.977005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:52.977099 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:52.977479 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:52.977482 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:52.977583 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 02:26:52.977706 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 02:26:52.977845 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":111,"threads_started":1}
I20260807 02:26:52.979306 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:26:52 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 02:26:53 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:53.008939 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:52.999653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.17.53.62:38273)
0807 02:26:52.999836 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:52.999839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:52.999854 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:26:52.999899 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:52.999901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:52.999945 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.000053 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:53.000888 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.001435 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.002388 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.002622 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.002744 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.005552 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:53.005557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:53.005562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:53.005595 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:53.007114 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:53.007573 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:53.007577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:53.007578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:53.007663 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:53.008010 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:53.008014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:53.008016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:53.008204 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:53.008350 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.008602 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.008738 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 02:26:53.011979 20268 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:53.015167 20268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:53.030041 21035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.023951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.0.0.1:47634)
0807 02:26:53.024885 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.024947 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.025085 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.025871 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.025881 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.025909 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.026051 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.026060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027169 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.027175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.028400 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.028412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.029501 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.029504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.029609 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.029646 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.029769 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.029890 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260807 02:26:53.030050 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.024833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.17.53.3:38833)
0807 02:26:53.025277 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.025290 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.025316 (+ 26us) server_negotiation.cc:407] Connection header received
0807 02:26:53.025399 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.025406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.025640 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.025856 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.026207 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027044 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.028544 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.029376 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.029499 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.029680 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.029806 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.029896 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":109,"threads_started":1}
I20260807 02:26:53.030745 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.023917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39559 (local address 127.0.0.1:60850)
0807 02:26:53.024885 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.024947 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.025085 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.026101 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.026106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.026129 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.026289 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.026297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027745 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.027749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.029011 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.029021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.030308 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.030312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.030416 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.030439 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.030515 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.030584 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":129,"spinlock_wait_cycles":896,"thread_start_us":85,"threads_started":1}
I20260807 02:26:53.030985 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.024825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.17.53.1:45239)
0807 02:26:53.025444 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.025450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.025702 (+ 252us) server_negotiation.cc:407] Connection header received
0807 02:26:53.025802 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.025806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.025992 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.026150 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.026476 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027604 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.029192 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.029961 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.030072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.030452 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.030753 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.030859 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":83,"threads_started":1}
I20260807 02:26:53.031281 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.024110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.17.53.2:39559)
0807 02:26:53.025418 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.025430 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.025457 (+ 27us) server_negotiation.cc:407] Connection header received
0807 02:26:53.025553 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.025559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.025797 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.025954 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.026422 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027592 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.029135 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.029961 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.030072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.030229 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.031003 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.031107 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":138,"threads_started":1}
I20260807 02:26:53.032060 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.024267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45239 (local address 127.0.0.1:48018)
0807 02:26:53.025576 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.025590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.025689 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.026134 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.026137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.026172 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.026318 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.026324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.027745 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.027749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.029081 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.029092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.030526 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.030530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.030607 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.030628 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.030702 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.030780 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"mutex_wait_us":69,"spinlock_wait_cycles":4096,"thread_start_us":119,"threads_started":1}
I20260807 02:26:53.033430 20907 tablet_service.cc:1467] Processing CreateTablet for tablet 586594828e144fdeb2392ae5f8bf759f (DEFAULT_TABLE table=test-table [id=5e1356db34d0488a80c91febb4cac577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.033825 20694 tablet_service.cc:1467] Processing CreateTablet for tablet 586594828e144fdeb2392ae5f8bf759f (DEFAULT_TABLE table=test-table [id=5e1356db34d0488a80c91febb4cac577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.034605 20907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586594828e144fdeb2392ae5f8bf759f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.034631 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586594828e144fdeb2392ae5f8bf759f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.034749 20481 tablet_service.cc:1467] Processing CreateTablet for tablet 586594828e144fdeb2392ae5f8bf759f (DEFAULT_TABLE table=test-table [id=5e1356db34d0488a80c91febb4cac577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.035679 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586594828e144fdeb2392ae5f8bf759f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.042043 21047 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Bootstrap starting.
I20260807 02:26:53.043298 21050 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Bootstrap starting.
I20260807 02:26:53.044230 21047 tablet_bootstrap.cc:654] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.045060 21047 log.cc:824] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:53.045111 21049 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Bootstrap starting.
I20260807 02:26:53.045140 21050 tablet_bootstrap.cc:654] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.045957 21050 log.cc:824] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:53.047066 21049 tablet_bootstrap.cc:654] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.047381 21047 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: No bootstrap required, opened a new log
I20260807 02:26:53.047586 21047 ts_tablet_manager.cc:1397] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:26:53.047873 21049 log.cc:824] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:53.047982 21050 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: No bootstrap required, opened a new log
I20260807 02:26:53.048158 21050 ts_tablet_manager.cc:1397] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 02:26:53.049892 21049 tablet_bootstrap.cc:492] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: No bootstrap required, opened a new log
I20260807 02:26:53.050079 21049 ts_tablet_manager.cc:1397] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:26:53.050858 21047 raft_consensus.cc:348] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.051138 21047 raft_consensus.cc:374] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.051198 21047 raft_consensus.cc:729] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.051662 21047 consensus_queue.cc:260] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.052007 21050 raft_consensus.cc:348] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.052248 21050 raft_consensus.cc:374] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.052338 21050 raft_consensus.cc:729] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.052721 21047 ts_tablet_manager.cc:1428] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260807 02:26:53.052798 21050 consensus_queue.cc:260] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.053134 20801 heartbeater.cc:503] Master 127.17.53.62:38273 was elected leader, sending a full tablet report...
I20260807 02:26:53.053740 21050 ts_tablet_manager.cc:1428] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:26:53.054219 21007 heartbeater.cc:503] Master 127.17.53.62:38273 was elected leader, sending a full tablet report...
I20260807 02:26:53.056311 21049 raft_consensus.cc:348] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.056725 21049 raft_consensus.cc:374] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.056829 21049 raft_consensus.cc:729] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96d4206efef4c98ab336eb89fcbe607, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.057242 21049 consensus_queue.cc:260] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.057837 20587 heartbeater.cc:503] Master 127.17.53.62:38273 was elected leader, sending a full tablet report...
I20260807 02:26:53.058100 21049 ts_tablet_manager.cc:1428] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 02:26:53.157445 21058 raft_consensus.cc:482] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:53.157744 21058 raft_consensus.cc:504] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.159091 21058 leader_election.cc:288] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833), c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239)
I20260807 02:26:53.163556 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.159430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.17.53.2:40781)
0807 02:26:53.159561 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.159578 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.159718 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.160068 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.160071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.160097 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.160264 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.160276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.161299 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.161304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.162383 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.162394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.163094 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.163098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.163217 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.163275 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.163360 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.163436 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260807 02:26:53.163637 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.159589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:40781 (local address 127.17.53.3:38833)
0807 02:26:53.159764 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.159771 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.159789 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:26:53.159851 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.159855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.159916 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.160007 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.160416 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.161127 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.162538 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.162956 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.163043 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.163097 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.163406 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.163495 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 02:26:53.164522 20941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586594828e144fdeb2392ae5f8bf759f" candidate_uuid: "22fd916b80944e1db828d3d5c6e65536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" is_pre_election: true
I20260807 02:26:53.164799 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.159431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45239 (local address 127.17.53.2:52491)
0807 02:26:53.160204 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.160218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.160333 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.160636 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.160640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.160661 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.161348 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.161358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.162205 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.162209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.163198 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.163209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.164283 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.164287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.164395 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.164421 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.164515 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.164597 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":369,"thread_start_us":150,"threads_started":1}
I20260807 02:26:53.164868 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.159609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:52491 (local address 127.17.53.1:45239)
0807 02:26:53.159789 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.159792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.160320 (+ 528us) server_negotiation.cc:407] Connection header received
0807 02:26:53.160434 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.160437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.160497 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.160572 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.161521 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.162058 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.163395 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.164082 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.164212 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.164266 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.164545 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.164654 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:26:53.165005 20941 raft_consensus.cc:2454] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fd916b80944e1db828d3d5c6e65536 in term 0.
I20260807 02:26:53.165565 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586594828e144fdeb2392ae5f8bf759f" candidate_uuid: "22fd916b80944e1db828d3d5c6e65536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607" is_pre_election: true
I20260807 02:26:53.165642 20636 leader_election.cc:302] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fd916b80944e1db828d3d5c6e65536, 4cf5f598d2c84f868e5a7a3b61354d25; no voters:
I20260807 02:26:53.165896 20518 raft_consensus.cc:2454] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22fd916b80944e1db828d3d5c6e65536 in term 0.
I20260807 02:26:53.166047 21058 raft_consensus.cc:2781] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:53.166150 21058 raft_consensus.cc:482] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:53.166249 21058 raft_consensus.cc:3037] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.167466 21058 raft_consensus.cc:504] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.167924 21058 leader_election.cc:288] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [CANDIDATE]: Term 1 election: Requested vote from peers 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833), c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239)
I20260807 02:26:53.168269 20941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586594828e144fdeb2392ae5f8bf759f" candidate_uuid: "22fd916b80944e1db828d3d5c6e65536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf5f598d2c84f868e5a7a3b61354d25"
I20260807 02:26:53.168409 20941 raft_consensus.cc:3037] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.168522 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586594828e144fdeb2392ae5f8bf759f" candidate_uuid: "22fd916b80944e1db828d3d5c6e65536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607"
I20260807 02:26:53.168678 20518 raft_consensus.cc:3037] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.169775 20941 raft_consensus.cc:2454] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fd916b80944e1db828d3d5c6e65536 in term 1.
I20260807 02:26:53.170120 20636 leader_election.cc:302] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fd916b80944e1db828d3d5c6e65536, 4cf5f598d2c84f868e5a7a3b61354d25; no voters:
I20260807 02:26:53.170310 21058 raft_consensus.cc:2781] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:53.170347 20518 raft_consensus.cc:2454] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22fd916b80944e1db828d3d5c6e65536 in term 1.
I20260807 02:26:53.170611 21058 raft_consensus.cc:686] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 1 LEADER]: Becoming Leader. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Running, Role: LEADER
I20260807 02:26:53.171018 21058 consensus_queue.cc:237] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } }
I20260807 02:26:53.174773 20265 catalog_manager.cc:5495] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "22fd916b80944e1db828d3d5c6e65536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } health_report { overall_health: UNKNOWN } } }
W20260807 02:26:53.215134 21008 tablet.cc:2369] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:26:53.234825 20803 tablet.cc:2369] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 02:26:53 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069612, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
W20260807 02:26:53.241107 20589 tablet.cc:2369] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:26:53.243708 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.234659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38273 (local address 127.0.0.1:33648)
0807 02:26:53.234987 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.235005 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.235156 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.235452 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.235455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.235881 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:53.235979 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.235985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.236714 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.236717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.237332 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.237341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.237855 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.237861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.237942 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.238575 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:53.238587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:53.240190 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:53.242148 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:53.242160 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:53.242167 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:53.242390 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:53.242678 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:53.242681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:53.242683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:53.242769 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:53.243138 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:53.243146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:53.243254 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.243370 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.243518 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":123,"threads_started":1}
I20260807 02:26:53.243708 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.234725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.17.53.62:38273)
0807 02:26:53.234871 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.234876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.235104 (+ 228us) server_negotiation.cc:407] Connection header received
0807 02:26:53.235263 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.235266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.235320 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.235416 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:53.236091 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.236620 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.237463 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.237680 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.237789 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.240342 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:53.240346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:53.240348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:53.240372 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:53.242035 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:53.242491 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:53.242496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:53.242498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:53.242575 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:53.242858 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:53.242862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:53.242864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:53.243030 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:53.243138 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.243418 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.243576 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":53}
I20260807 02:26:53.247836 20265 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:33648:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 02:26:53.248667 20265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:53.255203 20481 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b09fa966a470dad8631a8c56a0bf8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1342b1368ce4966ab17433150638b12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:53.255518 20907 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b09fa966a470dad8631a8c56a0bf8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1342b1368ce4966ab17433150638b12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:53.256129 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b09fa966a470dad8631a8c56a0bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.256278 20907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b09fa966a470dad8631a8c56a0bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.260097 21049 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: Bootstrap starting.
I20260807 02:26:53.260494 21050 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: Bootstrap starting.
I20260807 02:26:53.260535 20694 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6b09fa966a470dad8631a8c56a0bf8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1342b1368ce4966ab17433150638b12]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:26:53.261246 21049 tablet_bootstrap.cc:654] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.261332 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6b09fa966a470dad8631a8c56a0bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.261538 21050 tablet_bootstrap.cc:654] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.262990 21049 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: No bootstrap required, opened a new log
I20260807 02:26:53.263048 21050 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: No bootstrap required, opened a new log
I20260807 02:26:53.263095 21049 ts_tablet_manager.cc:1397] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 02:26:53.263150 21050 ts_tablet_manager.cc:1397] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:26:53.263703 21050 raft_consensus.cc:348] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.263823 21050 raft_consensus.cc:374] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.263875 21050 raft_consensus.cc:729] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.263729 21049 raft_consensus.cc:348] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.264015 21049 raft_consensus.cc:374] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.264070 21049 raft_consensus.cc:729] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96d4206efef4c98ab336eb89fcbe607, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.264030 21050 consensus_queue.cc:260] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.264278 21049 consensus_queue.cc:260] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.264353 21050 ts_tablet_manager.cc:1428] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:26:53.264317 21059 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.264460 21059 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.264585 21049 ts_tablet_manager.cc:1428] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 02:26:53.264575 21063 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.264736 21063 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.266927 21047 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: Bootstrap starting.
I20260807 02:26:53.268236 21047 tablet_bootstrap.cc:654] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.269914 21047 tablet_bootstrap.cc:492] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: No bootstrap required, opened a new log
I20260807 02:26:53.270018 21047 ts_tablet_manager.cc:1397] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:26:53.270556 21047 raft_consensus.cc:348] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.270990 21047 raft_consensus.cc:374] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.271080 21047 raft_consensus.cc:729] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.271246 21047 consensus_queue.cc:260] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.271556 21058 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.271731 21058 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.272460 21047 ts_tablet_manager.cc:1428] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:53.563087 21058 consensus_queue.cc:1035] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.569577 21058 consensus_queue.cc:1035] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.595225 21059 raft_consensus.cc:482] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:53.595373 21059 raft_consensus.cc:504] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.596524 21059 leader_election.cc:288] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:53.600600 21063 raft_consensus.cc:482] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:53.600735 21063 raft_consensus.cc:504] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.600893 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.596861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39559 (local address 127.17.53.3:34887)
0807 02:26:53.597220 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.597237 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.597378 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.597889 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.597892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.597911 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.598030 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.598036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.598780 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.598787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.599622 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.599631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.600312 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.600315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.600483 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.600501 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.600604 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.600685 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":48,"thread_start_us":93,"threads_started":1}
I20260807 02:26:53.600893 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.596992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34887 (local address 127.17.53.2:39559)
0807 02:26:53.597585 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.597589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.597601 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:26:53.597646 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.597648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.597729 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.597867 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.598148 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.598666 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.599759 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.600171 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.600288 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.600369 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.600643 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.600720 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":106,"threads_started":1}
I20260807 02:26:53.600965 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.596990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50865 (local address 127.17.53.1:45239)
0807 02:26:53.597143 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.597148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.597372 (+ 224us) server_negotiation.cc:407] Connection header received
0807 02:26:53.597487 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.597489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.597535 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.597626 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.597971 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.598588 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.599754 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.600154 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.600303 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.600374 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.600620 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.600713 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:26:53.601359 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.596861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45239 (local address 127.17.53.3:50865)
0807 02:26:53.597281 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.597295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.597379 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.597647 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.597650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.597670 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.597848 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.597855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.598714 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.598718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.599629 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.599639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.600312 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.600315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.600446 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.600481 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.601148 (+ 667us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.601215 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":48,"spinlock_wait_cycles":7936,"thread_start_us":66,"threads_started":1}
I20260807 02:26:53.601564 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536" is_pre_election: true
I20260807 02:26:53.601805 20705 raft_consensus.cc:2454] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:53.601790 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607" is_pre_election: true
I20260807 02:26:53.601908 20518 raft_consensus.cc:2454] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:53.602064 21063 leader_election.cc:288] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559), 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833)
I20260807 02:26:53.602264 20834 leader_election.cc:302] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:53.602552 21059 raft_consensus.cc:2781] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:53.602619 21059 raft_consensus.cc:482] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:53.602666 21059 raft_consensus.cc:3037] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.604058 21059 raft_consensus.cc:504] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.604786 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607"
I20260807 02:26:53.604735 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536"
I20260807 02:26:53.604897 20705 raft_consensus.cc:3037] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.604897 20518 raft_consensus.cc:3037] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.606055 20518 raft_consensus.cc:2454] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:53.606056 20705 raft_consensus.cc:2454] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:53.606395 21059 leader_election.cc:288] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Requested vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:53.606379 20834 leader_election.cc:302] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:53.606564 21059 raft_consensus.cc:2781] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:53.606757 21059 raft_consensus.cc:686] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Running, Role: LEADER
I20260807 02:26:53.606958 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.602742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:48771 (local address 127.17.53.3:38833)
0807 02:26:53.602864 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.602871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.603068 (+ 197us) server_negotiation.cc:407] Connection header received
0807 02:26:53.603161 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.603164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.603215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.603312 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.603631 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.604079 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.605695 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.606300 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.606379 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.606437 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.606790 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.606862 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:26:53.607012 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.602353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.17.53.1:48771)
0807 02:26:53.602968 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.602980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.603066 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.603415 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.603417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.603431 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.603536 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.603544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.604196 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.604200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.605541 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.605554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.606539 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.606543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.606649 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.606674 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.606755 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.606838 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":21,"thread_start_us":113,"threads_started":1}
I20260807 02:26:53.607102 21059 consensus_queue.cc:237] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.607455 20941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "c96d4206efef4c98ab336eb89fcbe607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" is_pre_election: true
I20260807 02:26:53.607528 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.602391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:47907 (local address 127.17.53.2:39559)
0807 02:26:53.602848 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.602852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.602962 (+ 110us) server_negotiation.cc:407] Connection header received
0807 02:26:53.603127 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.603130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.603179 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.603263 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:26:53.603553 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.604245 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.605752 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.606413 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.606564 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.606662 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.607337 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.607413 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377}
I20260807 02:26:53.607887 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.602240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39559 (local address 127.17.53.1:47907)
0807 02:26:53.602853 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.602868 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.602978 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.603308 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.603312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.603328 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:26:53.603428 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.603435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.604361 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.604364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.605624 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.605634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.606758 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.606762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.606904 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.606927 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.607646 (+ 719us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.607738 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":23,"thread_start_us":146,"threads_started":1}
I20260807 02:26:53.608294 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6b09fa966a470dad8631a8c56a0bf8" candidate_uuid: "c96d4206efef4c98ab336eb89fcbe607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536" is_pre_election: true
I20260807 02:26:53.608409 21059 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: TxnStatusTablet state changed. Reason: New leader 4cf5f598d2c84f868e5a7a3b61354d25. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.608469 20705 raft_consensus.cc:2379] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c96d4206efef4c98ab336eb89fcbe607 in current term 1: Already voted for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in this term.
I20260807 02:26:53.608505 21059 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:53.608975 20413 leader_election.cc:302] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c96d4206efef4c98ab336eb89fcbe607; no voters: 22fd916b80944e1db828d3d5c6e65536, 4cf5f598d2c84f868e5a7a3b61354d25
I20260807 02:26:53.609257 21063 raft_consensus.cc:2726] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 02:26:53.609295 21100 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 02:26:53.609596 21100 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 02:26:53.610004 20265 catalog_manager.cc:5495] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } health_report { overall_health: HEALTHY } } }
I20260807 02:26:53.642498 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.638043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.0.0.1:47636)
0807 02:26:53.638245 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.638283 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.638389 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.638791 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.638793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.638802 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:53.638907 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.638911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.640115 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.640121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.640831 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.640847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.641337 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.641340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.641436 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.641483 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.642180 (+ 697us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.642200 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.642286 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.642375 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 02:26:53.642498 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.638113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.17.53.3:38833)
0807 02:26:53.638292 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.638297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.638316 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:26:53.638548 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.638552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.638623 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.638755 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.639040 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.639952 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.640968 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.641217 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.641331 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.641639 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.642072 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.642205 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.642303 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.642355 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 02:26:53.648290 20518 raft_consensus.cc:1261] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:53.648464 20705 raft_consensus.cc:1261] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:53.649006 21098 consensus_queue.cc:1035] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.649243 21059 consensus_queue.cc:1035] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.652451 21058 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: TxnStatusTablet state changed. Reason: New leader 4cf5f598d2c84f868e5a7a3b61354d25. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.652598 21058 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.653921 21063 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: TxnStatusTablet state changed. Reason: New leader 4cf5f598d2c84f868e5a7a3b61354d25. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.654045 21063 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.656112 21059 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.656246 21059 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:53.656497 21082 mvcc.cc:205] Tried to move back new op lower bound from 7315741137496834048 to 7315741137338122240. Current Snapshot: MvccSnapshot[applied={T|T < 7315741137496834048}]
I20260807 02:26:53.656692 21063 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.656826 21063 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P c96d4206efef4c98ab336eb89fcbe607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.656800 21098 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.656888 21098 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 4cf5f598d2c84f868e5a7a3b61354d25: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:26:53.657527 21058 tablet_replica.cc:439] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } } }
I20260807 02:26:53.657644 21058 tablet_replica.cc:442] T 6f6b09fa966a470dad8631a8c56a0bf8 P 22fd916b80944e1db828d3d5c6e65536: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:26:53.658085 21079 mvcc.cc:205] Tried to move back new op lower bound from 7315741137496834048 to 7315741137338122240. Current Snapshot: MvccSnapshot[applied={T|T < 7315741137496834048}]
I20260807 02:26:53.668983 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.664651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.17.53.2:39559)
0807 02:26:53.664801 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.664806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.665025 (+ 219us) server_negotiation.cc:407] Connection header received
0807 02:26:53.665146 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.665150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.665237 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.665324 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.665626 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.666234 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.667459 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.667671 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.667754 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.668030 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.668497 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.668593 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.668703 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.668813 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:26:53.680279 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.676800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.17.53.2:39677)
0807 02:26:53.677119 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.677150 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.677276 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.677526 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.677529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.677537 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:53.677648 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.677653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.678401 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.678405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.679023 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.679031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.679500 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.679504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.679590 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.679640 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.679991 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.680000 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.680067 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.680146 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":108,"threads_started":1}
I20260807 02:26:53.680277 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.676960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:39677 (local address 127.17.53.3:38833)
0807 02:26:53.677089 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.677094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.677249 (+ 155us) server_negotiation.cc:407] Connection header received
0807 02:26:53.677366 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.677369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.677425 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.677505 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.677762 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.678276 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.679159 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.679388 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.679497 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.679821 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.679901 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.679985 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.680076 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.680128 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:26:53.710767 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.706978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34115 (local address 127.17.53.2:39559)
0807 02:26:53.707120 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.707124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.707140 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:26:53.707192 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.707195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.707247 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.707334 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.707500 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.708280 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.709725 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.709945 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.710027 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.710368 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.710440 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.710494 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.710607 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.710654 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 02:26:53.711336 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.706600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39559 (local address 127.17.53.3:34115)
0807 02:26:53.707004 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.707021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.707127 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.707337 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.707339 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.707344 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:53.707431 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.707435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.708416 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.708419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.709546 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.709560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.710062 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.710066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.710155 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.710221 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.710523 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.710530 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.710600 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.710689 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":159,"threads_started":1}
I20260807 02:26:53.741761 20265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:53.741981 20265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:53.745527 20265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Sending DeleteTablet for 3 replicas of tablet 586594828e144fdeb2392ae5f8bf759f
I20260807 02:26:53.746816 20694 tablet_service.cc:1514] Processing DeleteTablet for tablet 586594828e144fdeb2392ae5f8bf759f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:53 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60850
I20260807 02:26:53.747380 20907 tablet_service.cc:1514] Processing DeleteTablet for tablet 586594828e144fdeb2392ae5f8bf759f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:53 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47634
I20260807 02:26:53.748313 20481 tablet_service.cc:1514] Processing DeleteTablet for tablet 586594828e144fdeb2392ae5f8bf759f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:53 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48018
I20260807 02:26:53.749073 20265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:53.749531 20265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:53.752986 21125 tablet_replica.cc:330] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: stopping tablet replica
I20260807 02:26:53.753437 21125 raft_consensus.cc:2229] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:53.753993 21125 raft_consensus.cc:2258] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:53.755322 20481 tablet_service.cc:1467] Processing CreateTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a (DEFAULT_TABLE table=test-table [id=86f1a60c36d4439c8770f9b283f82358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.755647 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75f9b9bc136467cb7b2d99b93e67e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.756266 21125 ts_tablet_manager.cc:1905] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:53.757033 20907 tablet_service.cc:1467] Processing CreateTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a (DEFAULT_TABLE table=test-table [id=86f1a60c36d4439c8770f9b283f82358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.757323 20907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75f9b9bc136467cb7b2d99b93e67e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.759801 21050 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Bootstrap starting.
I20260807 02:26:53.760980 21128 tablet_replica.cc:330] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: stopping tablet replica
I20260807 02:26:53.761233 21050 tablet_bootstrap.cc:654] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.761340 21128 raft_consensus.cc:2229] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:53.761597 21128 raft_consensus.cc:2258] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:53.762557 20694 tablet_service.cc:1467] Processing CreateTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a (DEFAULT_TABLE table=test-table [id=86f1a60c36d4439c8770f9b283f82358]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:53.762830 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75f9b9bc136467cb7b2d99b93e67e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:53.763283 21128 ts_tablet_manager.cc:1905] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:53.763859 21125 ts_tablet_manager.cc:1918] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:53.764520 21125 log.cc:1196] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/586594828e144fdeb2392ae5f8bf759f
I20260807 02:26:53.765183 21125 ts_tablet_manager.cc:1939] T 586594828e144fdeb2392ae5f8bf759f P 22fd916b80944e1db828d3d5c6e65536: Deleting consensus metadata
I20260807 02:26:53.765343 21047 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Bootstrap starting.
I20260807 02:26:53.766420 21047 tablet_bootstrap.cc:654] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.766682 21050 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: No bootstrap required, opened a new log
I20260807 02:26:53.766804 21050 ts_tablet_manager.cc:1397] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260807 02:26:53.766948 20246 catalog_manager.cc:4841] TS 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559): tablet 586594828e144fdeb2392ae5f8bf759f (table test-table [id=5e1356db34d0488a80c91febb4cac577]) successfully deleted
I20260807 02:26:53.767418 21050 raft_consensus.cc:348] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.767565 21050 raft_consensus.cc:374] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.767606 21050 raft_consensus.cc:729] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.767776 21050 consensus_queue.cc:260] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.768229 21050 ts_tablet_manager.cc:1428] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 02:26:53.769143 21128 ts_tablet_manager.cc:1918] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:53.769304 21128 log.cc:1196] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/586594828e144fdeb2392ae5f8bf759f
I20260807 02:26:53.769677 21128 ts_tablet_manager.cc:1939] T 586594828e144fdeb2392ae5f8bf759f P c96d4206efef4c98ab336eb89fcbe607: Deleting consensus metadata
I20260807 02:26:53.770815 21047 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: No bootstrap required, opened a new log
I20260807 02:26:53.770908 21047 ts_tablet_manager.cc:1397] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 02:26:53.771354 21047 raft_consensus.cc:348] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.771497 21047 raft_consensus.cc:374] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.771528 21047 raft_consensus.cc:729] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.771592 20242 catalog_manager.cc:4841] TS c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239): tablet 586594828e144fdeb2392ae5f8bf759f (table test-table [id=5e1356db34d0488a80c91febb4cac577]) successfully deleted
I20260807 02:26:53.771646 21047 consensus_queue.cc:260] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.771896 21047 ts_tablet_manager.cc:1428] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:53.773754 21132 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Bootstrap starting.
I20260807 02:26:53.774811 21132 tablet_bootstrap.cc:654] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:53.775439 21126 tablet_replica.cc:330] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: stopping tablet replica
I20260807 02:26:53.775789 21126 raft_consensus.cc:2229] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:53.775969 21126 raft_consensus.cc:2258] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:53.778052 21132 tablet_bootstrap.cc:492] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: No bootstrap required, opened a new log
I20260807 02:26:53.778142 21132 ts_tablet_manager.cc:1397] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 02:26:53.778631 21132 raft_consensus.cc:348] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.778754 21132 raft_consensus.cc:374] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:53.778787 21132 raft_consensus.cc:729] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96d4206efef4c98ab336eb89fcbe607, State: Initialized, Role: FOLLOWER
I20260807 02:26:53.779110 21126 ts_tablet_manager.cc:1905] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:53.782325 21126 ts_tablet_manager.cc:1918] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:53.782439 21126 log.cc:1196] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/586594828e144fdeb2392ae5f8bf759f
I20260807 02:26:53.782781 21126 ts_tablet_manager.cc:1939] T 586594828e144fdeb2392ae5f8bf759f P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting consensus metadata
I20260807 02:26:53.784025 20249 catalog_manager.cc:4841] TS 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833): tablet 586594828e144fdeb2392ae5f8bf759f (table test-table [id=5e1356db34d0488a80c91febb4cac577]) successfully deleted
I20260807 02:26:53.784257 21132 consensus_queue.cc:260] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.784600 21132 ts_tablet_manager.cc:1428] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260807 02:26:53.900604 21059 raft_consensus.cc:482] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:53.900826 21059 raft_consensus.cc:504] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.901376 21059 leader_election.cc:288] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:53.901891 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75f9b9bc136467cb7b2d99b93e67e0a" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536" is_pre_election: true
I20260807 02:26:53.901877 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75f9b9bc136467cb7b2d99b93e67e0a" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607" is_pre_election: true
I20260807 02:26:53.902063 20705 raft_consensus.cc:2454] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:53.902063 20518 raft_consensus.cc:2454] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:53.902416 20837 leader_election.cc:302] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fd916b80944e1db828d3d5c6e65536, 4cf5f598d2c84f868e5a7a3b61354d25; no voters:
I20260807 02:26:53.902616 21059 raft_consensus.cc:2781] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:53.902696 21059 raft_consensus.cc:482] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:53.902753 21059 raft_consensus.cc:3037] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.903761 21059 raft_consensus.cc:504] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.904124 21059 leader_election.cc:288] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Requested vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:53.904502 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75f9b9bc136467cb7b2d99b93e67e0a" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607"
I20260807 02:26:53.904547 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75f9b9bc136467cb7b2d99b93e67e0a" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536"
I20260807 02:26:53.904636 20518 raft_consensus.cc:3037] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.904701 20705 raft_consensus.cc:3037] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:53.905624 20518 raft_consensus.cc:2454] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:53.905900 20705 raft_consensus.cc:2454] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:53.905972 20834 leader_election.cc:302] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:53.906144 21059 raft_consensus.cc:2781] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:53.906230 21059 raft_consensus.cc:686] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Running, Role: LEADER
I20260807 02:26:53.906378 21059 consensus_queue.cc:237] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:53.908356 20268 catalog_manager.cc:5495] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } health_report { overall_health: HEALTHY } } }
I20260807 02:26:53.941087 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.937617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.17.53.3:38833)
0807 02:26:53.937785 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.937793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.937900 (+ 107us) server_negotiation.cc:407] Connection header received
0807 02:26:53.938076 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.938080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.938130 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.938212 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.938479 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.939087 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.939952 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.940172 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.940293 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.940554 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.940626 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.940710 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.940873 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.940934 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 02:26:53.950686 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.947440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:51579 (local address 127.17.53.3:38833)
0807 02:26:53.947565 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:53.947568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:53.947582 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:26:53.947732 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:53.947735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:53.947777 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:53.947851 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:53.948068 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.948816 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.949669 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.949928 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.950016 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.950240 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.950311 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.950381 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:53.950508 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 02:26:53.950551 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:26:53.950688 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:53.947344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38833 (local address 127.17.53.3:51579)
0807 02:26:53.947527 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:53.947545 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:53.947613 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:53.947877 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:53.947879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:53.947886 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:26:53.947971 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.947974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.948949 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.948952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.949535 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:53.949544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:53.950035 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:53.950037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:53.950101 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:53.950127 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:26:53.950428 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:26:53.950435 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 02:26:53.950507 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 02:26:53.950578 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 02:26:53.956768 20705 raft_consensus.cc:1261] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:53.957238 21059 consensus_queue.cc:1035] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.957283 20518 raft_consensus.cc:1261] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:53.957657 21059 consensus_queue.cc:1035] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:53.960834 21088 mvcc.cc:205] Tried to move back new op lower bound from 7315741138763063296 to 7315741138561761280. Current Snapshot: MvccSnapshot[applied={T|T < 7315741138763063296}]
W20260807 02:26:53.974493 21088 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 02:26:55.534652 20265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:55.534807 20265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:55.537776 20265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Sending DeleteTablet for 3 replicas of tablet d75f9b9bc136467cb7b2d99b93e67e0a
I20260807 02:26:55.538550 20907 tablet_service.cc:1514] Processing DeleteTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47634
I20260807 02:26:55.538547 20481 tablet_service.cc:1514] Processing DeleteTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48018
I20260807 02:26:55.538583 20694 tablet_service.cc:1514] Processing DeleteTablet for tablet d75f9b9bc136467cb7b2d99b93e67e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60850
I20260807 02:26:55.538926 21181 tablet_replica.cc:330] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: stopping tablet replica
I20260807 02:26:55.538924 21182 tablet_replica.cc:330] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: stopping tablet replica
I20260807 02:26:55.539050 21181 raft_consensus.cc:2229] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:55.539050 21182 raft_consensus.cc:2229] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:55.539160 21181 raft_consensus.cc:2258] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.539160 21182 raft_consensus.cc:2258] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.539872 21182 ts_tablet_manager.cc:1905] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.543040 21182 ts_tablet_manager.cc:1918] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 02:26:55.543139 21182 log.cc:1196] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/d75f9b9bc136467cb7b2d99b93e67e0a
I20260807 02:26:55.543360 21181 ts_tablet_manager.cc:1905] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.543601 21182 ts_tablet_manager.cc:1939] T d75f9b9bc136467cb7b2d99b93e67e0a P c96d4206efef4c98ab336eb89fcbe607: Deleting consensus metadata
I20260807 02:26:55.544405 20242 catalog_manager.cc:4841] TS c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239): tablet d75f9b9bc136467cb7b2d99b93e67e0a (table test-table [id=86f1a60c36d4439c8770f9b283f82358]) successfully deleted
I20260807 02:26:55.546144 21181 ts_tablet_manager.cc:1918] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 02:26:55.546242 21181 log.cc:1196] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/d75f9b9bc136467cb7b2d99b93e67e0a
I20260807 02:26:55.546523 21181 ts_tablet_manager.cc:1939] T d75f9b9bc136467cb7b2d99b93e67e0a P 22fd916b80944e1db828d3d5c6e65536: Deleting consensus metadata
I20260807 02:26:55.547334 21183 tablet_replica.cc:330] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: stopping tablet replica
I20260807 02:26:55.547386 20246 catalog_manager.cc:4841] TS 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559): tablet d75f9b9bc136467cb7b2d99b93e67e0a (table test-table [id=86f1a60c36d4439c8770f9b283f82358]) successfully deleted
I20260807 02:26:55.547487 21183 raft_consensus.cc:2229] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:55.547809 21183 raft_consensus.cc:2258] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.548414 21183 ts_tablet_manager.cc:1905] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.550592 21183 ts_tablet_manager.cc:1918] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 02:26:55.550686 21183 log.cc:1196] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/d75f9b9bc136467cb7b2d99b93e67e0a
I20260807 02:26:55.551036 21183 ts_tablet_manager.cc:1939] T d75f9b9bc136467cb7b2d99b93e67e0a P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting consensus metadata
I20260807 02:26:55.551754 20249 catalog_manager.cc:4841] TS 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833): tablet d75f9b9bc136467cb7b2d99b93e67e0a (table test-table [id=86f1a60c36d4439c8770f9b283f82358]) successfully deleted
Aug 07 02:26:55 dist-test-slave-g170 krb5kdc[20197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 02:26:55 dist-test-slave-g170 krb5kdc[20197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:55.573357 21194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:55.558907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.17.53.62:38273)
0807 02:26:55.559178 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:55.559189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:55.559277 (+ 88us) server_negotiation.cc:407] Connection header received
0807 02:26:55.559464 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:55.559467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:55.559510 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:55.559602 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:55.560293 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:55.560976 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:55.561840 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:55.562083 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:55.562201 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:55.569890 (+ 7689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:55.569894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:55.569897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:55.569924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:55.571583 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:55.571987 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:55.571989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:55.571991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:55.572077 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:55.572391 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:55.572394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:55.572395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:55.572591 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:55.572705 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:55.572995 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 02:26:55.573140 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":88,"threads_started":1}
I20260807 02:26:55.576021 20265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:55.576428 20265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:55.581557 20481 tablet_service.cc:1467] Processing CreateTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e (DEFAULT_TABLE table=test-table [id=4660f051bac6430dbf617c45be85ff73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.581663 20694 tablet_service.cc:1467] Processing CreateTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e (DEFAULT_TABLE table=test-table [id=4660f051bac6430dbf617c45be85ff73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.581786 20907 tablet_service.cc:1467] Processing CreateTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e (DEFAULT_TABLE table=test-table [id=4660f051bac6430dbf617c45be85ff73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.581908 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd10d6f5dfde4a48868859bb512e3e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.581984 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd10d6f5dfde4a48868859bb512e3e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.582082 20907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd10d6f5dfde4a48868859bb512e3e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.583982 21195 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Bootstrap starting.
I20260807 02:26:55.585035 21197 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Bootstrap starting.
I20260807 02:26:55.585081 21195 tablet_bootstrap.cc:654] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.585351 21200 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Bootstrap starting.
I20260807 02:26:55.586125 21197 tablet_bootstrap.cc:654] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.586323 21200 tablet_bootstrap.cc:654] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.586619 21195 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: No bootstrap required, opened a new log
I20260807 02:26:55.586714 21195 ts_tablet_manager.cc:1397] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:26:55.587319 21195 raft_consensus.cc:348] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.587455 21195 raft_consensus.cc:374] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.587496 21195 raft_consensus.cc:729] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96d4206efef4c98ab336eb89fcbe607, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.587584 21200 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: No bootstrap required, opened a new log
I20260807 02:26:55.587678 21200 ts_tablet_manager.cc:1397] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:55.587661 21195 consensus_queue.cc:260] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.588053 21195 ts_tablet_manager.cc:1428] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:55.588121 21200 raft_consensus.cc:348] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.588234 21200 raft_consensus.cc:374] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.588277 21200 raft_consensus.cc:729] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.588418 21200 consensus_queue.cc:260] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.588831 21200 ts_tablet_manager.cc:1428] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 02:26:55.589534 21197 tablet_bootstrap.cc:492] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: No bootstrap required, opened a new log
I20260807 02:26:55.589694 21197 ts_tablet_manager.cc:1397] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 02:26:55.590303 21197 raft_consensus.cc:348] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.590504 21197 raft_consensus.cc:374] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.590603 21197 raft_consensus.cc:729] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.590839 21197 consensus_queue.cc:260] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.591264 21197 ts_tablet_manager.cc:1428] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:55.602250 21160 raft_consensus.cc:482] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:55.602387 21160 raft_consensus.cc:504] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.602770 21160 leader_election.cc:288] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:55.603138 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd10d6f5dfde4a48868859bb512e3e5e" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536" is_pre_election: true
I20260807 02:26:55.603143 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd10d6f5dfde4a48868859bb512e3e5e" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607" is_pre_election: true
I20260807 02:26:55.603298 20518 raft_consensus.cc:2454] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:55.603324 20705 raft_consensus.cc:2454] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:55.603652 20834 leader_election.cc:302] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:55.603857 21160 raft_consensus.cc:2781] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:55.603924 21160 raft_consensus.cc:482] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:55.603967 21160 raft_consensus.cc:3037] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.605012 21160 raft_consensus.cc:504] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.605369 21160 leader_election.cc:288] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Requested vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:55.605700 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd10d6f5dfde4a48868859bb512e3e5e" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607"
I20260807 02:26:55.605705 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd10d6f5dfde4a48868859bb512e3e5e" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536"
I20260807 02:26:55.605826 20518 raft_consensus.cc:3037] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.605836 20705 raft_consensus.cc:3037] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.606693 20705 raft_consensus.cc:2454] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:55.606778 20518 raft_consensus.cc:2454] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:55.607004 20837 leader_election.cc:302] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22fd916b80944e1db828d3d5c6e65536, 4cf5f598d2c84f868e5a7a3b61354d25; no voters:
I20260807 02:26:55.607208 21160 raft_consensus.cc:2781] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:55.607321 21160 raft_consensus.cc:686] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Running, Role: LEADER
I20260807 02:26:55.607486 21160 consensus_queue.cc:237] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.609144 20268 catalog_manager.cc:5495] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } health_report { overall_health: HEALTHY } } }
I20260807 02:26:55.626827 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:55.623569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.17.53.3:38833)
0807 02:26:55.623896 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:55.623904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:55.623915 (+ 11us) server_negotiation.cc:407] Connection header received
0807 02:26:55.623954 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:55.623956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:55.623996 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:55.624066 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:26:55.624292 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:55.624943 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:55.625848 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:55.626068 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:55.626148 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:55.626376 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:26:55.626445 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:26:55.626501 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:55.626589 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 02:26:55.626632 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260807 02:26:55.632831 20705 raft_consensus.cc:1261] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:55.632931 20518 raft_consensus.cc:1261] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:55.633317 21059 consensus_queue.cc:1035] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:55.633515 21160 consensus_queue.cc:1035] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:55.663002 20265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:55.663151 20265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:55.665508 20265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Sending DeleteTablet for 3 replicas of tablet bd10d6f5dfde4a48868859bb512e3e5e
I20260807 02:26:55.666157 20481 tablet_service.cc:1514] Processing DeleteTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48018
I20260807 02:26:55.666213 20694 tablet_service.cc:1514] Processing DeleteTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60850
I20260807 02:26:55.666230 20904 tablet_service.cc:1514] Processing DeleteTablet for tablet bd10d6f5dfde4a48868859bb512e3e5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:55 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47634
I20260807 02:26:55.666415 21183 tablet_replica.cc:330] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: stopping tablet replica
I20260807 02:26:55.666427 21182 tablet_replica.cc:330] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: stopping tablet replica
I20260807 02:26:55.666546 21182 raft_consensus.cc:2229] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:55.666548 21183 raft_consensus.cc:2229] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:55.666687 21182 raft_consensus.cc:2258] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.666754 21183 raft_consensus.cc:2258] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.666914 21181 tablet_replica.cc:330] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: stopping tablet replica
I20260807 02:26:55.667054 21181 raft_consensus.cc:2229] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:55.667192 21181 raft_consensus.cc:2258] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:55.667585 21182 ts_tablet_manager.cc:1905] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.667946 21181 ts_tablet_manager.cc:1905] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.668382 21183 ts_tablet_manager.cc:1905] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:55.668650 20265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:26:55.669353 20265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:26:55.670467 21182 ts_tablet_manager.cc:1918] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:55.670548 21182 log.cc:1196] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/bd10d6f5dfde4a48868859bb512e3e5e
I20260807 02:26:55.670830 21182 ts_tablet_manager.cc:1939] T bd10d6f5dfde4a48868859bb512e3e5e P c96d4206efef4c98ab336eb89fcbe607: Deleting consensus metadata
I20260807 02:26:55.670913 21181 ts_tablet_manager.cc:1918] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:55.671010 21181 log.cc:1196] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/bd10d6f5dfde4a48868859bb512e3e5e
I20260807 02:26:55.671331 21183 ts_tablet_manager.cc:1918] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:26:55.671394 21181 ts_tablet_manager.cc:1939] T bd10d6f5dfde4a48868859bb512e3e5e P 22fd916b80944e1db828d3d5c6e65536: Deleting consensus metadata
I20260807 02:26:55.671420 21183 log.cc:1196] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/bd10d6f5dfde4a48868859bb512e3e5e
I20260807 02:26:55.671489 20242 catalog_manager.cc:4841] TS c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239): tablet bd10d6f5dfde4a48868859bb512e3e5e (table test-table [id=4660f051bac6430dbf617c45be85ff73]) successfully deleted
I20260807 02:26:55.671689 21183 ts_tablet_manager.cc:1939] T bd10d6f5dfde4a48868859bb512e3e5e P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting consensus metadata
I20260807 02:26:55.672122 20246 catalog_manager.cc:4841] TS 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559): tablet bd10d6f5dfde4a48868859bb512e3e5e (table test-table [id=4660f051bac6430dbf617c45be85ff73]) successfully deleted
I20260807 02:26:55.672361 20249 catalog_manager.cc:4841] TS 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833): tablet bd10d6f5dfde4a48868859bb512e3e5e (table test-table [id=4660f051bac6430dbf617c45be85ff73]) successfully deleted
I20260807 02:26:55.676455 20481 tablet_service.cc:1467] Processing CreateTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed (DEFAULT_TABLE table=test-table [id=e5f364971b7b46f3937dc766959ba772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.676748 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6277f9cd359c4b2da3872d6fb59af6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.676707 20904 tablet_service.cc:1467] Processing CreateTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed (DEFAULT_TABLE table=test-table [id=e5f364971b7b46f3937dc766959ba772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.677006 20904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6277f9cd359c4b2da3872d6fb59af6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.677181 20694 tablet_service.cc:1467] Processing CreateTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed (DEFAULT_TABLE table=test-table [id=e5f364971b7b46f3937dc766959ba772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:26:55.677438 20694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6277f9cd359c4b2da3872d6fb59af6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:55.679292 21200 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Bootstrap starting.
I20260807 02:26:55.679724 21197 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Bootstrap starting.
I20260807 02:26:55.680145 21200 tablet_bootstrap.cc:654] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.680437 21197 tablet_bootstrap.cc:654] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.680755 21195 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607: Bootstrap starting.
I20260807 02:26:55.681412 21200 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: No bootstrap required, opened a new log
I20260807 02:26:55.681506 21200 ts_tablet_manager.cc:1397] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 02:26:55.681779 21195 tablet_bootstrap.cc:654] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:55.681948 21200 raft_consensus.cc:348] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.682076 21200 raft_consensus.cc:374] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.682117 21200 raft_consensus.cc:729] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fd916b80944e1db828d3d5c6e65536, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.682264 21200 consensus_queue.cc:260] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.682600 21200 ts_tablet_manager.cc:1428] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:55.683100 21195 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607: No bootstrap required, opened a new log
I20260807 02:26:55.683104 21197 tablet_bootstrap.cc:492] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: No bootstrap required, opened a new log
I20260807 02:26:55.683180 21195 ts_tablet_manager.cc:1397] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:55.683180 21197 ts_tablet_manager.cc:1397] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 02:26:55.683564 21197 raft_consensus.cc:348] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.683574 21195 raft_consensus.cc:348] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.683670 21197 raft_consensus.cc:374] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.683670 21195 raft_consensus.cc:374] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:55.683704 21197 raft_consensus.cc:729] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.683704 21195 raft_consensus.cc:729] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96d4206efef4c98ab336eb89fcbe607, State: Initialized, Role: FOLLOWER
I20260807 02:26:55.683840 21195 consensus_queue.cc:260] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.683862 21197 consensus_queue.cc:260] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.684108 21195 ts_tablet_manager.cc:1428] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:26:55.684108 21197 ts_tablet_manager.cc:1428] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:26:55.690502 21169 raft_consensus.cc:482] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:26:55.690591 21169 raft_consensus.cc:504] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.690900 21169 leader_election.cc:288] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:55.691205 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6277f9cd359c4b2da3872d6fb59af6ed" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607" is_pre_election: true
I20260807 02:26:55.691244 20724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6277f9cd359c4b2da3872d6fb59af6ed" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536" is_pre_election: true
I20260807 02:26:55.691358 20518 raft_consensus.cc:2454] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:55.691371 20724 raft_consensus.cc:2454] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 0.
I20260807 02:26:55.691630 20834 leader_election.cc:302] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:55.691866 21169 raft_consensus.cc:2781] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:26:55.691931 21169 raft_consensus.cc:482] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:26:55.691958 21169 raft_consensus.cc:3037] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.693001 21169 raft_consensus.cc:504] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.693308 21169 leader_election.cc:288] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Requested vote from peers c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239), 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559)
I20260807 02:26:55.693672 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6277f9cd359c4b2da3872d6fb59af6ed" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96d4206efef4c98ab336eb89fcbe607"
I20260807 02:26:55.693691 20724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6277f9cd359c4b2da3872d6fb59af6ed" candidate_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22fd916b80944e1db828d3d5c6e65536"
I20260807 02:26:55.693817 20518 raft_consensus.cc:3037] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.693837 20724 raft_consensus.cc:3037] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:55.694653 20518 raft_consensus.cc:2454] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:55.694979 20834 leader_election.cc:302] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cf5f598d2c84f868e5a7a3b61354d25, c96d4206efef4c98ab336eb89fcbe607; no voters:
I20260807 02:26:55.694981 20724 raft_consensus.cc:2454] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cf5f598d2c84f868e5a7a3b61354d25 in term 1.
I20260807 02:26:55.695142 21169 raft_consensus.cc:2781] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:55.695236 21169 raft_consensus.cc:686] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Becoming Leader. State: Replica: 4cf5f598d2c84f868e5a7a3b61354d25, State: Running, Role: LEADER
I20260807 02:26:55.695374 21169 consensus_queue.cc:237] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } }
I20260807 02:26:55.697036 20265 catalog_manager.cc:5495] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf5f598d2c84f868e5a7a3b61354d25" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38833 } health_report { overall_health: HEALTHY } } }
I20260807 02:26:55.716455 20724 raft_consensus.cc:1261] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:55.716499 20518 raft_consensus.cc:1261] T 6277f9cd359c4b2da3872d6fb59af6ed P c96d4206efef4c98ab336eb89fcbe607 [term 1 FOLLOWER]: Refusing update from remote peer 4cf5f598d2c84f868e5a7a3b61354d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:26:55.716881 21169 consensus_queue.cc:1035] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22fd916b80944e1db828d3d5c6e65536" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:26:55.717033 21144 consensus_queue.cc:1035] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96d4206efef4c98ab336eb89fcbe607" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 02:26:55.731168 21147 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 02:26:57.287472 20265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:57.287627 20265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:26:57.290242 20265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8290323db0c74aaa9bd401737a71b90e: Sending DeleteTablet for 3 replicas of tablet 6277f9cd359c4b2da3872d6fb59af6ed
I20260807 02:26:57.290963 20694 tablet_service.cc:1514] Processing DeleteTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:57 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60850
I20260807 02:26:57.290966 20481 tablet_service.cc:1514] Processing DeleteTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:57 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48018
I20260807 02:26:57.291301 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 20389
I20260807 02:26:57.291319 20904 tablet_service.cc:1514] Processing DeleteTablet for tablet 6277f9cd359c4b2da3872d6fb59af6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:26:57 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47634
I20260807 02:26:57.291318 21264 tablet_replica.cc:330] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: stopping tablet replica
I20260807 02:26:57.313789 21264 raft_consensus.cc:2229] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:26:57.313865 21267 tablet_replica.cc:330] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: stopping tablet replica
I20260807 02:26:57.314034 21267 raft_consensus.cc:2229] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:26:57.314289 21267 raft_consensus.cc:2258] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:57.314270 21264 raft_consensus.cc:2258] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:26:57.315404 21264 ts_tablet_manager.cc:1905] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:57.315526 21267 ts_tablet_manager.cc:1905] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:26:57.318043 21267 ts_tablet_manager.cc:1918] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 02:26:57.318156 21267 log.cc:1196] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/6277f9cd359c4b2da3872d6fb59af6ed
I20260807 02:26:57.318475 21267 ts_tablet_manager.cc:1939] T 6277f9cd359c4b2da3872d6fb59af6ed P 4cf5f598d2c84f868e5a7a3b61354d25: Deleting consensus metadata
I20260807 02:26:57.319108 21264 ts_tablet_manager.cc:1918] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 02:26:57.319226 21264 log.cc:1196] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/6277f9cd359c4b2da3872d6fb59af6ed
I20260807 02:26:57.319346 20249 catalog_manager.cc:4841] TS 4cf5f598d2c84f868e5a7a3b61354d25 (127.17.53.3:38833): tablet 6277f9cd359c4b2da3872d6fb59af6ed (table test-table [id=e5f364971b7b46f3937dc766959ba772]) successfully deleted
I20260807 02:26:57.319571 21264 ts_tablet_manager.cc:1939] T 6277f9cd359c4b2da3872d6fb59af6ed P 22fd916b80944e1db828d3d5c6e65536: Deleting consensus metadata
I20260807 02:26:57.320411 20246 catalog_manager.cc:4841] TS 22fd916b80944e1db828d3d5c6e65536 (127.17.53.2:39559): tablet 6277f9cd359c4b2da3872d6fb59af6ed (table test-table [id=e5f364971b7b46f3937dc766959ba772]) successfully deleted
W20260807 02:26:57.323982 20242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:45239 (error 108)
I20260807 02:26:57.324338 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 20603
I20260807 02:26:57.329008 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:57.328344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45239 (local address 127.0.0.1:34384)
0807 02:26:57.328689 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:57.328863 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
W20260807 02:26:57.329203 20242 catalog_manager.cc:4584] TS c96d4206efef4c98ab336eb89fcbe607 (127.17.53.1:45239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6277f9cd359c4b2da3872d6fb59af6ed: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45239: connect: Connection refused (error 111)
I20260807 02:26:57.335779 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 20813
I20260807 02:26:57.350682 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 20220
2026-08-07T02:26:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6715 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:57 dist-test-slave-g170 krb5kdc[21272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:57 dist-test-slave-g170 krb5kdc[21272](info): set up 2 sockets
Aug 07 02:26:57 dist-test-slave-g170 krb5kdc[21272](info): commencing operation
krb5kdc: starting...
W20260807 02:26:58.604417 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:26:58 dist-test-slave-g170 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:26:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:26:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:26:58.736372 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44001
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:44001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:58.851585 21288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:58.851863 21288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:58.851904 21288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:58.855939 21288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:26:58.855996 21288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:58.856015 21288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:58.856031 21288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:26:58.856046 21288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:26:58.861442 21288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:44001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:58.862458 21288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:58.863476 21288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:58.872359 21304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:58.872366 21298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:58.872366 21297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:58.872498 21288 server_base.cc:1034] running on GCE node
I20260807 02:26:58.873310 21288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:58.874393 21288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:58.875576 21288 hybrid_clock.cc:648] HybridClock initialized: now 1786069618875544 us; error 44 us; skew 500 ppm
Aug 07 02:26:58 dist-test-slave-g170 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069618, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:58.878526 21288 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:26:58.879556 21288 webserver.cc:460] Webserver started at http://127.17.53.62:46205/ using document root <none> and password file <none>
I20260807 02:26:58.880208 21288 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:58.880311 21288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:58.880576 21288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:58.882860 21288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "2f5fb2ee152c4716a84081bd6fda739c"
format_stamp: "Formatted at 2026-08-07 02:26:58 on dist-test-slave-g170"
server_key: "6db8e84507030fa5da6314050a4c77b6"
server_key_iv: "2161fe801fb74ab738ee125a56c26f94"
server_key_version: "encryptionkey@0"
I20260807 02:26:58.883486 21288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "2f5fb2ee152c4716a84081bd6fda739c"
format_stamp: "Formatted at 2026-08-07 02:26:58 on dist-test-slave-g170"
server_key: "6db8e84507030fa5da6314050a4c77b6"
server_key_iv: "2161fe801fb74ab738ee125a56c26f94"
server_key_version: "encryptionkey@0"
I20260807 02:26:58.887662 21288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260807 02:26:58.890414 21314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:58.891661 21288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:26:58.891817 21288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "2f5fb2ee152c4716a84081bd6fda739c"
format_stamp: "Formatted at 2026-08-07 02:26:58 on dist-test-slave-g170"
server_key: "6db8e84507030fa5da6314050a4c77b6"
server_key_iv: "2161fe801fb74ab738ee125a56c26f94"
server_key_version: "encryptionkey@0"
I20260807 02:26:58.891935 21288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:58.910058 21288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:58.913203 21288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:58.913416 21288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:58.921015 21288 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:44001
I20260807 02:26:58.921030 21400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:44001 every 8 connection(s)
I20260807 02:26:58.922075 21288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:26:58.923430 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 21288
I20260807 02:26:58.923563 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:26:58.923866 17620 external_mini_cluster.cc:1428] Setting key 4792c26f2d29258ff0493e2f20665d9c
I20260807 02:26:58.925271 21401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:26:58.930932 21401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c: Bootstrap starting.
Aug 07 02:26:58 dist-test-slave-g170 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:58.933645 21401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c: Neither blocks nor log segments found. Creating new log.
I20260807 02:26:58.934605 21401 log.cc:824] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c: Log is configured to *not* fsync() on all Append() calls
I20260807 02:26:58.937170 21401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c: No bootstrap required, opened a new log
I20260807 02:26:58.938721 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:58.925361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.17.53.62:44001)
0807 02:26:58.925855 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:58.925875 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:58.925913 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:26:58.926555 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:58.926572 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:58.926901 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:58.927242 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:58.928070 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:58.929156 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:58.930054 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:58.930545 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:58.930754 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:58.933509 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:58.933515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:58.933528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:58.933557 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:58.935638 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:58.936228 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:58.936233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:58.936238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:58.936351 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:58.936692 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:58.936695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:58.936696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:58.937152 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:58.937352 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:58.937637 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 02:26:58.937985 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":136,"threads_started":1}
I20260807 02:26:58.941056 21401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } }
I20260807 02:26:58.941279 21401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:26:58.941375 21401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5fb2ee152c4716a84081bd6fda739c, State: Initialized, Role: FOLLOWER
I20260807 02:26:58.941829 21401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } }
I20260807 02:26:58.941960 21401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:26:58.942052 21401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:26:58.942168 21401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:26:58.943159 21401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } }
I20260807 02:26:58.943570 21401 leader_election.cc:302] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f5fb2ee152c4716a84081bd6fda739c; no voters:
I20260807 02:26:58.943903 21401 leader_election.cc:288] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:26:58.944083 21411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:26:58.944379 21411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [term 1 LEADER]: Becoming Leader. State: Replica: 2f5fb2ee152c4716a84081bd6fda739c, State: Running, Role: LEADER
I20260807 02:26:58.944831 21411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } }
I20260807 02:26:58.945318 21401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:26:58.946766 21413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f5fb2ee152c4716a84081bd6fda739c. Latest consensus state: current_term: 1 leader_uuid: "2f5fb2ee152c4716a84081bd6fda739c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } } }
I20260807 02:26:58.946841 21412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f5fb2ee152c4716a84081bd6fda739c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f5fb2ee152c4716a84081bd6fda739c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44001 } } }
I20260807 02:26:58.946962 21412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:58.946908 21413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c [sys.catalog]: This master's current role is: LEADER
I20260807 02:26:58.947429 21422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:26:58.950955 21422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:26:58.956257 21422 catalog_manager.cc:1347] Generated new cluster ID: 04a03acbc9de464e9e4a38805ed3840c
I20260807 02:26:58.956336 21422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:26:58.970305 21422 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:26:58.971285 21422 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:26:58.981575 21422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f5fb2ee152c4716a84081bd6fda739c: Generated new TSK 0
I20260807 02:26:58.982249 21422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:26:59.018337 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:59.138154 21438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:59.138453 21438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:59.138499 21438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:59.142684 21438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:59.142761 21438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:59.142822 21438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:26:59.148663 21438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:59.149813 21438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:59.150856 21438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:59.160864 21445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:59.160864 21444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:59.160866 21452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:59.161374 21438 server_base.cc:1034] running on GCE node
I20260807 02:26:59.161824 21438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:59.162503 21438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:59.163687 21438 hybrid_clock.cc:648] HybridClock initialized: now 1786069619163664 us; error 34 us; skew 500 ppm
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:59.166932 21438 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:26:59.167851 21438 webserver.cc:460] Webserver started at http://127.17.53.1:43815/ using document root <none> and password file <none>
I20260807 02:26:59.168442 21438 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:59.168509 21438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:59.168748 21438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:59.170893 21438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "b1a229dd290241a2b2535567e88179bd"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "4b288fa8b5be5634d1434941f118d7f7"
server_key_iv: "cab05837c5d62408dfd9f30ecf8d40b7"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.171474 21438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "b1a229dd290241a2b2535567e88179bd"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "4b288fa8b5be5634d1434941f118d7f7"
server_key_iv: "cab05837c5d62408dfd9f30ecf8d40b7"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.175297 21438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 02:26:59.177789 21462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.178988 21438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:26:59.179126 21438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "b1a229dd290241a2b2535567e88179bd"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "4b288fa8b5be5634d1434941f118d7f7"
server_key_iv: "cab05837c5d62408dfd9f30ecf8d40b7"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.179247 21438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:59.202337 21438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:59.205760 21438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:59.205972 21438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:59.206578 21438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:59.207429 21438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:59.207501 21438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.207576 21438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:59.207628 21438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.217170 21438 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:39833
I20260807 02:26:59.217212 21629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:39833 every 8 connection(s)
I20260807 02:26:59.218160 21438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:26:59.226841 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 21438
I20260807 02:26:59.226976 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:26:59.227277 17620 external_mini_cluster.cc:1428] Setting key 6102a5829f947c1efb69636bdb32fddd
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:59.233222 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.220358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49423 (local address 127.17.53.62:44001)
0807 02:26:59.220540 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:59.220544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:59.221701 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 02:26:59.222702 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:59.222706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:59.222772 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:59.222868 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:59.224118 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.224672 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.225891 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.226134 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.226257 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.229036 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:59.229040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:59.229041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:59.229065 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:59.230613 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.231201 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.231209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.231211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.231286 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.231704 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.231707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.231708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.232113 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:59.232267 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:59.232912 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 02:26:59.233048 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 02:26:59.234072 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.220806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44001 (local address 127.17.53.1:49423)
0807 02:26:59.221539 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:59.221583 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:59.222438 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:59.223059 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:59.223066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:59.223472 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:59.223967 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.223981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.224834 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.224840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.225746 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.225757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.226317 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.226321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.226493 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.227174 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:59.227190 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:59.228861 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:59.230782 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.230789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.230805 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.231080 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.231387 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.231391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.231393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.231542 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.232294 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:59.232301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:59.232584 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 02:26:59.232854 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 02:26:59.233203 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":125,"threads_started":1}
I20260807 02:26:59.235428 21630 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44001
I20260807 02:26:59.235693 21630 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:59.236212 21630 heartbeater.cc:511] Master 127.17.53.62:44001 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:26:59.237977 21349 ts_manager.cc:194] Registered new tserver with Master: b1a229dd290241a2b2535567e88179bd (127.17.53.1:39833)
I20260807 02:26:59.239267 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:49423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:26:59.260082 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:59.377548 21640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:59.377794 21640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:59.377835 21640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:59.381826 21640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:59.381875 21640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:59.381964 21640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:26:59.387177 21640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:59.388198 21640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:59.389206 21640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:59.399216 21647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:59.399258 21646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:59.399375 21640 server_base.cc:1034] running on GCE node
W20260807 02:26:59.399258 21651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:59.399931 21640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:59.400586 21640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:59.401765 21640 hybrid_clock.cc:648] HybridClock initialized: now 1786069619401751 us; error 30 us; skew 500 ppm
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:59.404493 21640 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:26:59.405452 21640 webserver.cc:460] Webserver started at http://127.17.53.2:38045/ using document root <none> and password file <none>
I20260807 02:26:59.406051 21640 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:59.406160 21640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:59.406508 21640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:59.408581 21640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "5d0937b918ce4ee6920bc481d692cc37"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "f1de2f55659d17292e4967e3dedf77c9"
server_key_iv: "299f608eaf93692673c8ccd7669e4c5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.409236 21640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "5d0937b918ce4ee6920bc481d692cc37"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "f1de2f55659d17292e4967e3dedf77c9"
server_key_iv: "299f608eaf93692673c8ccd7669e4c5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.412964 21640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:26:59.415464 21660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.416743 21640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:26:59.416929 21640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "5d0937b918ce4ee6920bc481d692cc37"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "f1de2f55659d17292e4967e3dedf77c9"
server_key_iv: "299f608eaf93692673c8ccd7669e4c5b"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.417030 21640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:59.438764 21640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:59.442389 21640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:59.442624 21640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:59.443343 21640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:59.444284 21640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:59.444334 21640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.444402 21640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:59.444435 21640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.454216 21640 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:45631
I20260807 02:26:59.454238 21825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:45631 every 8 connection(s)
I20260807 02:26:59.455152 21640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:26:59.457697 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 21640
I20260807 02:26:59.457798 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:26:59.458097 17620 external_mini_cluster.cc:1428] Setting key dbf4057f4fb73d0304634dc9f4f55de3
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:26:59.470909 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.457388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:41151 (local address 127.17.53.62:44001)
0807 02:26:59.457590 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:59.457595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:59.458769 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 02:26:59.460116 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:59.460122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:59.460196 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:59.460290 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:59.461722 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.462302 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.463814 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.464045 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.464161 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.466736 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:59.466742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:59.466746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:59.466784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:59.468466 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.469125 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.469133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.469135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.469224 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.469807 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.469810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.469812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.470005 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:59.470089 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:59.470640 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 02:26:59.470783 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 02:26:59.471796 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.457989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44001 (local address 127.17.53.2:41151)
0807 02:26:59.458551 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:59.458607 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:59.459896 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:59.460491 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:59.460498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:59.460985 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:59.461501 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.461516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.462441 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.462447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.463675 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.463686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.464208 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.464212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.464401 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.465063 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:59.465080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:59.466547 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:59.468589 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.468594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.468608 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.468981 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.469440 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.469443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.469445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.469603 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.470117 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:59.470122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:59.470368 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 02:26:59.470528 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 02:26:59.470799 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":187,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:26:59.473244 21827 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44001
I20260807 02:26:59.473524 21827 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:59.474169 21827 heartbeater.cc:511] Master 127.17.53.62:44001 requested a full tablet report, sending...
I20260807 02:26:59.475415 21349 ts_manager.cc:194] Registered new tserver with Master: 5d0937b918ce4ee6920bc481d692cc37 (127.17.53.2:45631)
I20260807 02:26:59.475997 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:41151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:26:59.493865 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:26:59.610460 21837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:26:59.610756 21837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:26:59.610846 21837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:26:59.614874 21837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:26:59.614969 21837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:26:59.615064 21837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:26:59.620520 21837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:26:59.621819 21837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:26:59.622920 21837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:26:59.633330 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:59.633337 21850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:26:59.633368 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:26:59.633787 21837 server_base.cc:1034] running on GCE node
I20260807 02:26:59.634243 21837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:26:59.635032 21837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:26:59.636263 21837 hybrid_clock.cc:648] HybridClock initialized: now 1786069619636239 us; error 44 us; skew 500 ppm
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:26:59.639560 21837 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:26:59.640543 21837 webserver.cc:460] Webserver started at http://127.17.53.3:40837/ using document root <none> and password file <none>
I20260807 02:26:59.641279 21837 fs_manager.cc:359] Metadata directory not provided
I20260807 02:26:59.641352 21837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:26:59.641546 21837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:26:59.643787 21837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "70e5bd0fdc0a4ad0b61173d081919269"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "c92a3b9e869f16be3287caff0f6cdc82"
server_key_iv: "3d7d0bd143e9b034f693cbe5832bb5ed"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.644349 21837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "70e5bd0fdc0a4ad0b61173d081919269"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "c92a3b9e869f16be3287caff0f6cdc82"
server_key_iv: "3d7d0bd143e9b034f693cbe5832bb5ed"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.649276 21837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 02:26:59.652325 21859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.653803 21837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:26:59.653916 21837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "70e5bd0fdc0a4ad0b61173d081919269"
format_stamp: "Formatted at 2026-08-07 02:26:59 on dist-test-slave-g170"
server_key: "c92a3b9e869f16be3287caff0f6cdc82"
server_key_iv: "3d7d0bd143e9b034f693cbe5832bb5ed"
server_key_version: "encryptionkey@0"
I20260807 02:26:59.654057 21837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:26:59.692507 21837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:26:59.696157 21837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:26:59.696398 21837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:26:59.697045 21837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:26:59.697896 21837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:26:59.697945 21837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.697988 21837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:26:59.698011 21837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:26:59.707350 21837 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:34689
I20260807 02:26:59.707381 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:34689 every 8 connection(s)
I20260807 02:26:59.708272 21837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:26:59.712587 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 21837
I20260807 02:26:59.712715 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:26:59.713050 17620 external_mini_cluster.cc:1428] Setting key e30011b4acb53c9418ade0d52546f6a8
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069619, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:26:59.722940 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.710200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:49311 (local address 127.17.53.62:44001)
0807 02:26:59.710348 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:59.710352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:59.711170 (+ 818us) server_negotiation.cc:407] Connection header received
0807 02:26:59.712516 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:59.712521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:59.712585 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:59.712704 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:59.714218 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.714851 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.716136 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.716356 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.716436 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.718997 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:26:59.719001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:26:59.719003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:26:59.719032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:26:59.720552 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.721122 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.721125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.721126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.721217 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:26:59.721690 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:26:59.721692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:26:59.721693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:26:59.721891 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:26:59.721971 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:26:59.722618 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 02:26:59.722761 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:26:59.723939 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:26:59.710527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44001 (local address 127.17.53.3:49311)
0807 02:26:59.710998 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 02:26:59.711043 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:26:59.712268 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:26:59.712988 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:26:59.712998 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:26:59.713499 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:26:59.714037 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.714052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.714979 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.714983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.716019 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:26:59.716031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:26:59.716491 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:26:59.716494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:26:59.716633 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:26:59.717333 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:26:59.717350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:26:59.718818 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:26:59.720662 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.720667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.720680 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.721007 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.721352 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:26:59.721355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:26:59.721357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:26:59.721575 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:26:59.722018 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:26:59.722025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:26:59.722306 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 02:26:59.722550 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 02:26:59.722995 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":109,"threads_started":1}
I20260807 02:26:59.725436 22019 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44001
I20260807 02:26:59.725723 22019 heartbeater.cc:464] Registering TS with master...
I20260807 02:26:59.726264 22019 heartbeater.cc:511] Master 127.17.53.62:44001 requested a full tablet report, sending...
I20260807 02:26:59.727349 21349 ts_manager.cc:194] Registered new tserver with Master: 70e5bd0fdc0a4ad0b61173d081919269 (127.17.53.3:34689)
I20260807 02:26:59.727973 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:49311
I20260807 02:26:59.739446 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 02:26:59.756230 21404 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:26:59.754663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.17.53.62:44001)
0807 02:26:59.754875 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 02:26:59.754879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:26:59.754985 (+ 106us) server_negotiation.cc:407] Connection header received
0807 02:26:59.755074 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:26:59.755076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:26:59.755124 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:26:59.755218 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:26:59.756069 (+ 851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43600: BlockingRecv error: recv got EOF from 127.0.0.1:43600 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 02:26:59.757509 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 21438
I20260807 02:26:59.776937 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 21640
I20260807 02:26:59.800475 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 21837
I20260807 02:26:59.811211 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 21288
2026-08-07T02:26:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2455 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[22041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[22041](info): set up 2 sockets
Aug 07 02:26:59 dist-test-slave-g170 krb5kdc[22041](info): commencing operation
krb5kdc: starting...
W20260807 02:27:01.085546 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:01.221091 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:43795
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:43795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:01.336220 22059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:01.336508 22059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:01.336596 22059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:01.340605 22059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:01.340687 22059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:01.340724 22059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:01.340781 22059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:01.340821 22059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:01.346010 22059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:43795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:43795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:01.347095 22059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:01.348121 22059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:01.356685 22066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:01.356722 22070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:01.356987 22059 server_base.cc:1034] running on GCE node
W20260807 02:27:01.357105 22068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:01.357642 22059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:01.358731 22059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:01.359925 22059 hybrid_clock.cc:648] HybridClock initialized: now 1786069621359909 us; error 33 us; skew 500 ppm
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069621, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:01.368728 22059 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:01.369722 22059 webserver.cc:460] Webserver started at http://127.17.53.62:41261/ using document root <none> and password file <none>
I20260807 02:27:01.370292 22059 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:01.370385 22059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:01.370628 22059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:01.372650 22059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d0b25f4ea91a45e4b0058926075d3521"
format_stamp: "Formatted at 2026-08-07 02:27:01 on dist-test-slave-g170"
server_key: "58e4490b6cbc41911758e833297413dc"
server_key_iv: "566c91922dd5e3da9a7815747b158157"
server_key_version: "encryptionkey@0"
I20260807 02:27:01.373279 22059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d0b25f4ea91a45e4b0058926075d3521"
format_stamp: "Formatted at 2026-08-07 02:27:01 on dist-test-slave-g170"
server_key: "58e4490b6cbc41911758e833297413dc"
server_key_iv: "566c91922dd5e3da9a7815747b158157"
server_key_version: "encryptionkey@0"
I20260807 02:27:01.376948 22059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:01.379395 22079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:01.380568 22059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:01.380707 22059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d0b25f4ea91a45e4b0058926075d3521"
format_stamp: "Formatted at 2026-08-07 02:27:01 on dist-test-slave-g170"
server_key: "58e4490b6cbc41911758e833297413dc"
server_key_iv: "566c91922dd5e3da9a7815747b158157"
server_key_version: "encryptionkey@0"
I20260807 02:27:01.380878 22059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:01.402720 22059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:01.405962 22059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:01.406167 22059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:01.413714 22059 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:43795
I20260807 02:27:01.414000 22152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:43795 every 8 connection(s)
I20260807 02:27:01.414702 22059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:01.417573 22154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:01.418128 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22059
I20260807 02:27:01.418223 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:01.418536 17620 external_mini_cluster.cc:1428] Setting key 72ce632146966bbb3d72c219035e39f6
I20260807 02:27:01.424729 22154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: Bootstrap starting.
I20260807 02:27:01.426951 22154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: Neither blocks nor log segments found. Creating new log.
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:01.427748 22154 log.cc:824] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:01.429637 22154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: No bootstrap required, opened a new log
I20260807 02:27:01.432610 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:01.419890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.17.53.62:43795)
0807 02:27:01.420319 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:01.420340 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:01.420383 (+ 43us) server_negotiation.cc:407] Connection header received
0807 02:27:01.421166 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:01.421192 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:01.421623 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:01.421979 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:01.422939 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:01.424095 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:01.425038 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:01.425498 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:01.425667 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:01.428132 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:01.428137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:01.428150 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:01.428177 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:01.429956 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:01.430413 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:01.430418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:01.430422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:01.430532 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:01.430835 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:01.430838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:01.430839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:01.431198 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:01.431412 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:01.431672 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 02:27:01.431950 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260807 02:27:01.433224 22154 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } }
I20260807 02:27:01.433456 22154 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:01.433566 22154 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b25f4ea91a45e4b0058926075d3521, State: Initialized, Role: FOLLOWER
I20260807 02:27:01.434043 22154 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } }
I20260807 02:27:01.434204 22154 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:01.434273 22154 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:01.434412 22154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:01.435470 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:27:01.435552 22154 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } }
I20260807 02:27:01.435902 22154 leader_election.cc:302] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b25f4ea91a45e4b0058926075d3521; no voters:
I20260807 02:27:01.436220 22154 leader_election.cc:288] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:01.436368 22162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:01.436599 22162 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [term 1 LEADER]: Becoming Leader. State: Replica: d0b25f4ea91a45e4b0058926075d3521, State: Running, Role: LEADER
I20260807 02:27:01.437026 22162 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } }
I20260807 02:27:01.437456 22154 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:01.438800 22165 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b25f4ea91a45e4b0058926075d3521. Latest consensus state: current_term: 1 leader_uuid: "d0b25f4ea91a45e4b0058926075d3521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } } }
I20260807 02:27:01.438935 22165 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:01.439257 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b25f4ea91a45e4b0058926075d3521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b25f4ea91a45e4b0058926075d3521" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 43795 } } }
I20260807 02:27:01.439337 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521 [sys.catalog]: This master's current role is: LEADER
W20260807 02:27:01.443320 22186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:01.443413 22186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:01.443533 22177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:01.444604 22177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:01.450958 22177 catalog_manager.cc:1347] Generated new cluster ID: 537707dfc532440f99ae091a34ffa9d2
I20260807 02:27:01.451025 22177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:01.458079 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:01.452383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.17.53.62:43795)
0807 02:27:01.452489 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:01.452494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:01.452779 (+ 285us) server_negotiation.cc:407] Connection header received
0807 02:27:01.452973 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:01.452977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:01.453035 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:01.453129 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:01.453686 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:01.454534 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:01.455662 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:01.456044 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:01.456153 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:01.457074 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:01.457078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:01.457081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:01.457619 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:01.457655 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:01.457675 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:01.457771 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:01.457875 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 02:27:01.457936 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 02:27:01.463304 22177 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:01.464624 22177 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:01.470089 22177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b25f4ea91a45e4b0058926075d3521: Generated new TSK 0
I20260807 02:27:01.470850 22177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:01.475805 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22059
2026-08-07T02:27:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1662 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22205](info): set up 2 sockets
Aug 07 02:27:01 dist-test-slave-g170 krb5kdc[22205](info): commencing operation
krb5kdc: starting...
W20260807 02:27:02.727552 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:02 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:02.855834 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34059
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:34059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:02.968622 22220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:02.968912 22220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:02.968954 22220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:02.973105 22220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:02.973165 22220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:02.973186 22220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:02.973202 22220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:02.973217 22220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:02.978821 22220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:34059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:02.979921 22220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:02.981040 22220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:02.989760 22233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:02.989760 22228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:02.989778 22231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:02.990172 22220 server_base.cc:1034] running on GCE node
I20260807 02:27:02.990857 22220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:02.991925 22220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:02.993098 22220 hybrid_clock.cc:648] HybridClock initialized: now 1786069622993076 us; error 35 us; skew 500 ppm
Aug 07 02:27:02 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069622, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:02.996130 22220 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:02.997277 22220 webserver.cc:460] Webserver started at http://127.17.53.62:44505/ using document root <none> and password file <none>
I20260807 02:27:02.997949 22220 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:02.998021 22220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:02.998261 22220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:03.000547 22220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "2b8b99fde46644f58c71d5cd01f1999f"
format_stamp: "Formatted at 2026-08-07 02:27:02 on dist-test-slave-g170"
server_key: "2c7b8b4f9c96c6fe52b3d8b7d666d2d1"
server_key_iv: "9aaa4269716b6800ee73cf2b4a042c8b"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.001230 22220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "2b8b99fde46644f58c71d5cd01f1999f"
format_stamp: "Formatted at 2026-08-07 02:27:02 on dist-test-slave-g170"
server_key: "2c7b8b4f9c96c6fe52b3d8b7d666d2d1"
server_key_iv: "9aaa4269716b6800ee73cf2b4a042c8b"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.005352 22220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:03.008061 22242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.009513 22220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:03.009716 22220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "2b8b99fde46644f58c71d5cd01f1999f"
format_stamp: "Formatted at 2026-08-07 02:27:02 on dist-test-slave-g170"
server_key: "2c7b8b4f9c96c6fe52b3d8b7d666d2d1"
server_key_iv: "9aaa4269716b6800ee73cf2b4a042c8b"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.009826 22220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:03.045782 22220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:03.049602 22220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:03.049943 22220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:03.058189 22220 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:34059
I20260807 02:27:03.058429 22317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:34059 every 8 connection(s)
I20260807 02:27:03.059271 22220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:03.062203 22318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:03.062813 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22220
I20260807 02:27:03.062942 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:03.063252 17620 external_mini_cluster.cc:1428] Setting key 0651a165b6bcecd47899f29dfc4cf8fb
I20260807 02:27:03.069490 22318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f: Bootstrap starting.
I20260807 02:27:03.072438 22318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f: Neither blocks nor log segments found. Creating new log.
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:03.073314 22318 log.cc:824] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:03.075278 22318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f: No bootstrap required, opened a new log
I20260807 02:27:03.078992 22318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } }
I20260807 02:27:03.079247 22318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:03.079329 22318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8b99fde46644f58c71d5cd01f1999f, State: Initialized, Role: FOLLOWER
I20260807 02:27:03.079780 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.064694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.17.53.62:34059)
0807 02:27:03.065154 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:03.065170 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:03.065206 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:27:03.065901 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:03.065919 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:03.066251 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:03.066536 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:03.067476 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.068672 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.069896 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.070376 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.070499 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.073841 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:03.073851 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:03.073868 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:03.073901 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:03.076085 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.076693 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.076700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.076705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.076910 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.077278 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.077283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.077285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.077757 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:03.078016 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:03.078428 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 02:27:03.078765 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":120,"threads_started":1}
I20260807 02:27:03.079790 22318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } }
I20260807 02:27:03.080062 22318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:03.080149 22318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:03.080257 22318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:03.081360 22318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } }
I20260807 02:27:03.081710 22318 leader_election.cc:302] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b8b99fde46644f58c71d5cd01f1999f; no voters:
I20260807 02:27:03.082068 22318 leader_election.cc:288] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:03.082146 22325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:03.082361 22325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [term 1 LEADER]: Becoming Leader. State: Replica: 2b8b99fde46644f58c71d5cd01f1999f, State: Running, Role: LEADER
I20260807 02:27:03.082674 22325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } }
I20260807 02:27:03.083439 22318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:03.083921 22325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b8b99fde46644f58c71d5cd01f1999f. Latest consensus state: current_term: 1 leader_uuid: "2b8b99fde46644f58c71d5cd01f1999f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } } }
I20260807 02:27:03.084040 22325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:03.084270 22325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b8b99fde46644f58c71d5cd01f1999f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b8b99fde46644f58c71d5cd01f1999f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34059 } } }
I20260807 02:27:03.084376 22325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:03.084527 22334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:03.087481 22334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:03.093618 22334 catalog_manager.cc:1347] Generated new cluster ID: 2d8973c21e984a2699cb8b91e04827de
I20260807 02:27:03.093691 22334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:03.100865 22334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:03.101904 22334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:03.110424 22334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b8b99fde46644f58c71d5cd01f1999f: Generated new TSK 0
I20260807 02:27:03.111167 22334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:03.151513 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:03.269595 22360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:03.269855 22360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:03.269897 22360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:03.273890 22360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:03.273947 22360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:03.274034 22360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:03.279256 22360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:03.280274 22360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:03.281374 22360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:03.291318 22369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:03.291376 22360 server_base.cc:1034] running on GCE node
W20260807 02:27:03.291326 22366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:03.291326 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:03.291997 22360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:03.292580 22360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:03.293735 22360 hybrid_clock.cc:648] HybridClock initialized: now 1786069623293702 us; error 48 us; skew 500 ppm
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:03.300514 22360 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:03.301455 22360 webserver.cc:460] Webserver started at http://127.17.53.1:43581/ using document root <none> and password file <none>
I20260807 02:27:03.302076 22360 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:03.302188 22360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:03.302529 22360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:03.304958 22360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "35e1ec5027e540dcbb9be6d75a3f1e7d"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "8242175be2204f0016e27d739cf12355"
server_key_iv: "0c9034911caaea7690cd9030be277735"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.305555 22360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "35e1ec5027e540dcbb9be6d75a3f1e7d"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "8242175be2204f0016e27d739cf12355"
server_key_iv: "0c9034911caaea7690cd9030be277735"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.309473 22360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:03.312297 22385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.314158 22360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:03.314307 22360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "35e1ec5027e540dcbb9be6d75a3f1e7d"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "8242175be2204f0016e27d739cf12355"
server_key_iv: "0c9034911caaea7690cd9030be277735"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.314431 22360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:03.333107 22360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:03.337343 22360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:03.337568 22360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:03.338397 22360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:03.339573 22360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:03.339643 22360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.339704 22360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:03.339812 22360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.353593 22360 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:40347
I20260807 02:27:03.353724 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:40347 every 8 connection(s)
I20260807 02:27:03.354622 22360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:03.360416 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22360
I20260807 02:27:03.360558 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:03.360927 17620 external_mini_cluster.cc:1428] Setting key a8683d71c80a652a3cc85759b6db097f
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:03.372661 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.356853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:32935 (local address 127.17.53.62:34059)
0807 02:27:03.356996 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:03.357001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:03.358062 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 02:27:03.360169 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:03.360172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:03.360238 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:03.360341 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:03.362138 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.362887 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.364872 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.365114 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.365200 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.368753 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:03.368756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:03.368758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:03.368835 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:03.370283 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.370931 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.370939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.370940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.371019 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.371415 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.371418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.371420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.371608 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:03.371720 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:03.372347 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 02:27:03.372478 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:27:03.373776 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.357273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34059 (local address 127.17.53.1:32935)
0807 02:27:03.357874 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:03.357941 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:03.359870 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:03.360558 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:03.360570 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:03.361164 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:03.361941 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.361966 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.363070 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.363076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.364667 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.364682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.365297 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.365303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.365529 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.366268 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:03.366295 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:03.368507 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:03.370410 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.370417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.370432 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.370797 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.371118 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.371122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.371124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.371321 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.371720 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:03.371729 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:03.372054 (+ 325us) client_negotiation.cc:769] Sending connection context
0807 02:27:03.372279 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 02:27:03.372616 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":120,"threads_started":1}
I20260807 02:27:03.375053 22553 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34059
I20260807 02:27:03.375283 22553 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:03.375783 22553 heartbeater.cc:511] Master 127.17.53.62:34059 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:27:03.377597 22265 ts_manager.cc:194] Registered new tserver with Master: 35e1ec5027e540dcbb9be6d75a3f1e7d (127.17.53.1:40347)
I20260807 02:27:03.379011 22265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:32935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:03.395913 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:03.511078 22559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:03.511343 22559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:03.511386 22559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:03.515398 22559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:03.515461 22559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:03.515519 22559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:03.521221 22559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:03.522338 22559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:03.523380 22559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:03.533464 22570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:03.533464 22568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:03.533510 22565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:03.533727 22559 server_base.cc:1034] running on GCE node
I20260807 02:27:03.534261 22559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:03.534971 22559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:03.536154 22559 hybrid_clock.cc:648] HybridClock initialized: now 1786069623536105 us; error 59 us; skew 500 ppm
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:03.539376 22559 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:03.540452 22559 webserver.cc:460] Webserver started at http://127.17.53.2:44419/ using document root <none> and password file <none>
I20260807 02:27:03.541144 22559 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:03.541251 22559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:03.541505 22559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:03.543708 22559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "d9d6995bd28741148001d2ade1a63733"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "d2c8504ed0753e91e8a511b381a9d0af"
server_key_iv: "701252387dbc6c6bdb9c413e6cf4ff2f"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.544296 22559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "d9d6995bd28741148001d2ade1a63733"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "d2c8504ed0753e91e8a511b381a9d0af"
server_key_iv: "701252387dbc6c6bdb9c413e6cf4ff2f"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.548455 22559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 02:27:03.551153 22578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.552438 22559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 02:27:03.552577 22559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "d9d6995bd28741148001d2ade1a63733"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "d2c8504ed0753e91e8a511b381a9d0af"
server_key_iv: "701252387dbc6c6bdb9c413e6cf4ff2f"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.552726 22559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:03.570303 22559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:03.573338 22559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:03.573558 22559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:03.574236 22559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:03.575116 22559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:03.575201 22559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.575277 22559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:03.575313 22559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.585402 22559 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40251
I20260807 02:27:03.585434 22743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40251 every 8 connection(s)
I20260807 02:27:03.586380 22559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:03.593632 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22559
I20260807 02:27:03.593773 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:03.594075 17620 external_mini_cluster.cc:1428] Setting key f8e27a64fa5f14bbc28f3b99ab83fa85
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:03.603303 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.588538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:33077 (local address 127.17.53.62:34059)
0807 02:27:03.588711 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:03.588715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:03.589792 (+ 1077us) server_negotiation.cc:407] Connection header received
0807 02:27:03.591154 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:03.591158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:03.591211 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:03.591302 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:03.593009 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.593610 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.595191 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.595526 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.595647 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.598763 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:03.598770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:03.598775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:03.598813 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:03.600572 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.601338 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.601342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.601343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.601428 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.601957 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.601960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.601961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.602169 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:03.602252 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:03.603044 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 02:27:03.603169 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:27:03.604727 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.589078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34059 (local address 127.17.53.2:33077)
0807 02:27:03.589625 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:03.589671 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:03.590925 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:03.591476 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:03.591482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:03.592113 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:03.592738 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.592849 (+ 111us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.593796 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.593802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.595065 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.595075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.595737 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.595742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.595976 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.596652 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:03.596681 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:03.598554 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:03.600718 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.600728 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.600749 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.601185 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.601538 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.601544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.601549 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.601826 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.602290 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:03.602299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:03.602700 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 02:27:03.602966 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 02:27:03.603428 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":129,"threads_started":1}
I20260807 02:27:03.606066 22744 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34059
I20260807 02:27:03.606375 22744 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:27:03.607069 22744 heartbeater.cc:511] Master 127.17.53.62:34059 requested a full tablet report, sending...
I20260807 02:27:03.608354 22265 ts_manager.cc:194] Registered new tserver with Master: d9d6995bd28741148001d2ade1a63733 (127.17.53.2:40251)
I20260807 02:27:03.609197 22265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:33077
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:03.629458 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:03.751658 22752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:03.751967 22752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:03.752017 22752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:03.756349 22752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:03.756417 22752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:03.756477 22752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:03.762097 22752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:03.763226 22752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:03.764292 22752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:03.774168 22764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:03.774171 22759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:03.774204 22760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:03.774518 22752 server_base.cc:1034] running on GCE node
I20260807 02:27:03.774998 22752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:03.775662 22752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:03.776854 22752 hybrid_clock.cc:648] HybridClock initialized: now 1786069623776820 us; error 45 us; skew 500 ppm
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:03.779876 22752 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:03.780922 22752 webserver.cc:460] Webserver started at http://127.17.53.3:39507/ using document root <none> and password file <none>
I20260807 02:27:03.781569 22752 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:03.781687 22752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:03.781956 22752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:03.784530 22752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "ac7beb773cc24df6a3638a65f693df4f"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "b24c739a3edd5953fb41fb19df9eca47"
server_key_iv: "7ce71a729022171cd6a875377297dab7"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.785306 22752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "ac7beb773cc24df6a3638a65f693df4f"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "b24c739a3edd5953fb41fb19df9eca47"
server_key_iv: "7ce71a729022171cd6a875377297dab7"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.789603 22752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:03.792315 22775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.793833 22752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:27:03.794013 22752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "ac7beb773cc24df6a3638a65f693df4f"
format_stamp: "Formatted at 2026-08-07 02:27:03 on dist-test-slave-g170"
server_key: "b24c739a3edd5953fb41fb19df9eca47"
server_key_iv: "7ce71a729022171cd6a875377297dab7"
server_key_version: "encryptionkey@0"
I20260807 02:27:03.794147 22752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:03.806308 22752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:03.809592 22752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:03.809826 22752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:03.810509 22752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:03.811419 22752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:03.811544 22752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.811645 22752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:03.811707 22752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:03.822662 22752 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:43099
I20260807 02:27:03.822679 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:43099 every 8 connection(s)
I20260807 02:27:03.823771 22752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:03.828065 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22752
I20260807 02:27:03.828205 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:03.828547 17620 external_mini_cluster.cc:1428] Setting key 986659b014f77379d16bd133f5b4e06d
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:03.839359 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.826092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34605 (local address 127.17.53.62:34059)
0807 02:27:03.826283 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:03.826288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:03.827038 (+ 750us) server_negotiation.cc:407] Connection header received
0807 02:27:03.828506 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:03.828510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:03.828576 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:03.828682 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:03.830491 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.831304 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.832526 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.832754 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.832895 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.835565 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:03.835568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:03.835571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:03.835596 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:03.837183 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.837770 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.837773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.837775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.837859 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.838220 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.838223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.838224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.838418 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:03.838496 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:03.839078 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 02:27:03.839187 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 02:27:03.840163 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.826327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34059 (local address 127.17.53.3:34605)
0807 02:27:03.826868 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:03.826916 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:03.828235 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:03.828980 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:03.828989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:03.829623 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:03.830306 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.830336 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.831458 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.831461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.832400 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:03.832411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.832972 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.832976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:03.833148 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.833776 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:03.833794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:03.835394 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:03.837304 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.837311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.837328 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.837632 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.837975 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:03.837978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:03.837979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:03.838129 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:03.838524 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:03.838531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:03.838807 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 02:27:03.838978 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 02:27:03.839295 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":125,"threads_started":1}
I20260807 02:27:03.841415 22942 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34059
I20260807 02:27:03.841707 22942 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:03.842362 22942 heartbeater.cc:511] Master 127.17.53.62:34059 requested a full tablet report, sending...
I20260807 02:27:03.843603 22265 ts_manager.cc:194] Registered new tserver with Master: ac7beb773cc24df6a3638a65f693df4f (127.17.53.3:43099)
I20260807 02:27:03.844163 22265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:34605
I20260807 02:27:03.855019 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069623, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:03.883986 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:03.874938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.17.53.62:34059)
0807 02:27:03.875078 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:03.875082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:03.875263 (+ 181us) server_negotiation.cc:407] Connection header received
0807 02:27:03.875386 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:03.875389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:03.875435 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:03.875516 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:03.876347 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.876939 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.877846 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:03.878059 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:03.878142 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:03.880634 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:03.880637 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:03.880639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:03.880666 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:03.882192 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.882625 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.882627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.882629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.882704 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:03.882950 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:03.882952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:03.882953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:03.883182 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:03.883288 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:03.883683 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 02:27:03.883855 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 02:27:03.884445 22265 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59672
I20260807 02:27:03.885468 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22360
I20260807 02:27:03.895998 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22559
I20260807 02:27:03.911062 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22752
I20260807 02:27:03.921720 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22220
2026-08-07T02:27:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2452 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22959](info): set up 2 sockets
Aug 07 02:27:03 dist-test-slave-g170 krb5kdc[22959](info): commencing operation
krb5kdc: starting...
W20260807 02:27:05.237968 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:05 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:05.372035 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38473
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:38473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:05.486742 22978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:05.487023 22978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:05.487066 22978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:05.491235 22978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:05.491291 22978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:05.491310 22978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:05.491326 22978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:05.491341 22978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:05.496538 22978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:38473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:05.497659 22978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:05.498711 22978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:05.507607 22983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:05.507596 22984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:05.507616 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:05.507952 22978 server_base.cc:1034] running on GCE node
I20260807 02:27:05.508639 22978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:05.509802 22978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:05.510987 22978 hybrid_clock.cc:648] HybridClock initialized: now 1786069625510968 us; error 32 us; skew 500 ppm
Aug 07 02:27:05 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069625, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:05.513854 22978 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:05.514734 22978 webserver.cc:460] Webserver started at http://127.17.53.62:43527/ using document root <none> and password file <none>
I20260807 02:27:05.515318 22978 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:05.515412 22978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:05.515647 22978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:05.517756 22978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "2fdace0aa1b0436faf4d02238478a3da"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "43fdededffc64d3f078bd26b47638c17"
server_key_iv: "70001564f4697d618afb15e54c196208"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.518333 22978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "2fdace0aa1b0436faf4d02238478a3da"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "43fdededffc64d3f078bd26b47638c17"
server_key_iv: "70001564f4697d618afb15e54c196208"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.522136 22978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:05.524554 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:05.525852 22978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:05.526003 22978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "2fdace0aa1b0436faf4d02238478a3da"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "43fdededffc64d3f078bd26b47638c17"
server_key_iv: "70001564f4697d618afb15e54c196208"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.526124 22978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:05.542308 22978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:05.545658 22978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:05.545866 22978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:05.553356 22978 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:38473
I20260807 02:27:05.553372 23086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:38473 every 8 connection(s)
I20260807 02:27:05.554343 22978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:05.557299 23087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:05.559526 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 22978
I20260807 02:27:05.559664 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:05.559967 17620 external_mini_cluster.cc:1428] Setting key 69d7c7c7d5ec67152da1f8416d49a63d
I20260807 02:27:05.564146 23087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da: Bootstrap starting.
I20260807 02:27:05.566459 23087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:05.567364 23087 log.cc:824] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:27:05 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:05.569398 23087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da: No bootstrap required, opened a new log
I20260807 02:27:05.572729 23087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } }
I20260807 02:27:05.573024 23087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:05.573117 23087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdace0aa1b0436faf4d02238478a3da, State: Initialized, Role: FOLLOWER
I20260807 02:27:05.573593 23087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } }
I20260807 02:27:05.573715 23087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:05.573797 23087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:05.573892 23087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:05.574203 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:05.561439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.17.53.62:38473)
0807 02:27:05.561880 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:05.561895 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:05.561951 (+ 56us) server_negotiation.cc:407] Connection header received
0807 02:27:05.562621 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:05.562639 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:05.562969 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:05.563301 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:05.564113 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.565181 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.566021 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.566483 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.566663 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:05.569363 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:05.569368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:05.569380 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:05.569406 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:05.571130 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:05.571580 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:05.571591 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:05.571596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:05.571702 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:05.572003 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:05.572006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:05.572008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:05.572383 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:05.572590 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:05.572931 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 02:27:05.573367 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
I20260807 02:27:05.574944 23087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } }
I20260807 02:27:05.575304 23087 leader_election.cc:302] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fdace0aa1b0436faf4d02238478a3da; no voters:
I20260807 02:27:05.575673 23087 leader_election.cc:288] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:05.575754 23095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:05.575989 23095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [term 1 LEADER]: Becoming Leader. State: Replica: 2fdace0aa1b0436faf4d02238478a3da, State: Running, Role: LEADER
I20260807 02:27:05.576349 23095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } }
I20260807 02:27:05.577015 23087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:05.577651 23099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fdace0aa1b0436faf4d02238478a3da. Latest consensus state: current_term: 1 leader_uuid: "2fdace0aa1b0436faf4d02238478a3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } } }
I20260807 02:27:05.577870 23099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:05.578186 23096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fdace0aa1b0436faf4d02238478a3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdace0aa1b0436faf4d02238478a3da" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38473 } } }
I20260807 02:27:05.578285 23096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:05.578416 23105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:05.581596 23105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:05.587594 23105 catalog_manager.cc:1347] Generated new cluster ID: 218706de50984ffba396470c7dbf1617
I20260807 02:27:05.587661 23105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:05.594482 23105 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:05.595775 23105 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:05.603919 23105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fdace0aa1b0436faf4d02238478a3da: Generated new TSK 0
I20260807 02:27:05.604751 23105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:05.636650 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:05.758638 23127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:05.758929 23127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:05.758998 23127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:05.763263 23127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:05.763340 23127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:05.763428 23127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:05.768667 23127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:05.769776 23127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:05.770828 23127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:05.780786 23136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:05.780807 23133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:05.780948 23127 server_base.cc:1034] running on GCE node
W20260807 02:27:05.780797 23132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:05.781440 23127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:05.782171 23127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:05.783361 23127 hybrid_clock.cc:648] HybridClock initialized: now 1786069625783343 us; error 28 us; skew 500 ppm
Aug 07 02:27:05 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069625, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:05.786214 23127 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:05.787096 23127 webserver.cc:460] Webserver started at http://127.17.53.1:41369/ using document root <none> and password file <none>
I20260807 02:27:05.787662 23127 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:05.787750 23127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:05.787987 23127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:05.790093 23127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "603e4c76fc094054896e2617db128644"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "e8b18b7913cef958cc20b3e97a72ecab"
server_key_iv: "56028807966c26cac73e25300eeed25d"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.790655 23127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "603e4c76fc094054896e2617db128644"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "e8b18b7913cef958cc20b3e97a72ecab"
server_key_iv: "56028807966c26cac73e25300eeed25d"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.794361 23127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 02:27:05.796752 23145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:05.798076 23127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:27:05.798211 23127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "603e4c76fc094054896e2617db128644"
format_stamp: "Formatted at 2026-08-07 02:27:05 on dist-test-slave-g170"
server_key: "e8b18b7913cef958cc20b3e97a72ecab"
server_key_iv: "56028807966c26cac73e25300eeed25d"
server_key_version: "encryptionkey@0"
I20260807 02:27:05.798321 23127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:05.818661 23127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:05.822150 23127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:05.822363 23127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:05.822990 23127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:05.823872 23127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:05.823941 23127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:05.824013 23127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:05.824052 23127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:05.833587 23127 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:39587
I20260807 02:27:05.833628 23306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:39587 every 8 connection(s)
I20260807 02:27:05.834479 23127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:05.834880 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 23127
I20260807 02:27:05.835012 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:05.835386 17620 external_mini_cluster.cc:1428] Setting key c29ba15339e4d372e60a99c35058c681
Aug 07 02:27:05 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069625, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:27:05.850477 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:05.836300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:59713 (local address 127.17.53.62:38473)
0807 02:27:05.836448 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:05.836452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:05.837473 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 02:27:05.838879 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:05.838886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:05.838955 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:05.839067 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:05.840742 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.841689 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.842830 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.843257 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.843402 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:05.846249 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:05.846252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:05.846254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:05.846284 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:05.847964 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:05.848548 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:05.848551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:05.848553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:05.848640 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:05.849124 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:05.849127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:05.849128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:05.849392 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:05.849483 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:05.850158 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 02:27:05.850287 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:27:05.851351 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:05.836750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38473 (local address 127.17.53.1:59713)
0807 02:27:05.837284 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:05.837337 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:05.838655 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:05.839296 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:05.839307 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:05.839826 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:05.840558 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:05.840582 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.841845 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.841848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:05.842703 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:05.842713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:05.843415 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:05.843418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:05.843564 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:05.844180 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:05.844196 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:05.846063 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:05.848092 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:05.848098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:05.848111 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:05.848442 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:05.848852 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:05.848855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:05.848857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:05.849008 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:05.849505 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:05.849511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:05.849816 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 02:27:05.850078 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 02:27:05.850456 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":123,"threads_started":1}
I20260807 02:27:05.852792 23308 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38473
I20260807 02:27:05.853081 23308 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:05.853613 23308 heartbeater.cc:511] Master 127.17.53.62:38473 requested a full tablet report, sending...
I20260807 02:27:05.855177 23017 ts_manager.cc:194] Registered new tserver with Master: 603e4c76fc094054896e2617db128644 (127.17.53.1:39587)
I20260807 02:27:05.856333 23017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:59713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:05.867491 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:05.983961 23315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:05.984198 23315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:05.984241 23315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:05.988308 23315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:05.988361 23315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:05.988452 23315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:05.993597 23315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:05.994601 23315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:05.995610 23315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:06.005640 23322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:06.005640 23321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:06.005720 23315 server_base.cc:1034] running on GCE node
W20260807 02:27:06.005666 23325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:06.006299 23315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:06.007020 23315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:06.008242 23315 hybrid_clock.cc:648] HybridClock initialized: now 1786069626008217 us; error 70 us; skew 500 ppm
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:06.011471 23315 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:06.012419 23315 webserver.cc:460] Webserver started at http://127.17.53.2:40075/ using document root <none> and password file <none>
I20260807 02:27:06.013110 23315 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:06.013204 23315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:06.013447 23315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:06.015592 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "72171618cedc45b6b3ff173b498dc50a"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "2dab24f2ef8b417e1614d731fe17766c"
server_key_iv: "b2e1c096f0549ff3c95e85e59dd70957"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.016175 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "72171618cedc45b6b3ff173b498dc50a"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "2dab24f2ef8b417e1614d731fe17766c"
server_key_iv: "b2e1c096f0549ff3c95e85e59dd70957"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.020453 23315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 02:27:06.023363 23336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.025043 23315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 02:27:06.025195 23315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "72171618cedc45b6b3ff173b498dc50a"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "2dab24f2ef8b417e1614d731fe17766c"
server_key_iv: "b2e1c096f0549ff3c95e85e59dd70957"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.025336 23315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:06.041640 23315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:06.045162 23315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:06.045382 23315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:06.046053 23315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:06.046924 23315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:06.046973 23315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.047041 23315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:06.047079 23315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.056087 23315 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:38265
I20260807 02:27:06.056154 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:38265 every 8 connection(s)
I20260807 02:27:06.057066 23315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:06.065073 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 23315
I20260807 02:27:06.065212 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:06.065527 17620 external_mini_cluster.cc:1428] Setting key 07810ed8c5a16b543c3efd1bd43d5c46
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:06.071959 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.058924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:56273 (local address 127.17.53.62:38473)
0807 02:27:06.059061 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:06.059065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:06.059954 (+ 889us) server_negotiation.cc:407] Connection header received
0807 02:27:06.061489 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:06.061493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:06.061552 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:06.061657 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:06.062818 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.063397 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.064546 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.064830 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.064940 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.067954 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:06.067957 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:06.067959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:06.067983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:06.069525 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.070199 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.070203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.070204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.070299 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.070744 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.070747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.070748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.070946 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:06.071039 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:06.071727 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 02:27:06.071837 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:27:06.073390 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.059322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38473 (local address 127.17.53.2:56273)
0807 02:27:06.059803 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:06.059847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:06.061274 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:06.061842 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:06.061849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:06.062202 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:06.062674 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:06.062687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.063530 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.063533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:06.064418 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:06.064428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.064985 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.064989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:06.065154 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.065886 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:06.065903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:06.067766 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:06.069646 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:06.069656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:06.069674 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:06.070065 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:06.070406 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:06.070411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:06.070415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:06.070616 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:06.071066 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:06.071075 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:06.071429 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 02:27:06.071656 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 02:27:06.072032 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":116,"threads_started":1}
I20260807 02:27:06.074689 23514 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38473
I20260807 02:27:06.074965 23514 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:06.075479 23514 heartbeater.cc:511] Master 127.17.53.62:38473 requested a full tablet report, sending...
I20260807 02:27:06.076658 23017 ts_manager.cc:194] Registered new tserver with Master: 72171618cedc45b6b3ff173b498dc50a (127.17.53.2:38265)
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:27:06.077268 23017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:56273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:06.096706 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:06.224398 23519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:06.224689 23519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:06.224740 23519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:06.229295 23519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:06.229403 23519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:06.229497 23519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:06.235293 23519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:06.236371 23519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:06.237435 23519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:06.247327 23531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:06.247331 23525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:06.247371 23524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:06.247745 23519 server_base.cc:1034] running on GCE node
I20260807 02:27:06.248206 23519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:06.248951 23519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:06.250121 23519 hybrid_clock.cc:648] HybridClock initialized: now 1786069626250098 us; error 36 us; skew 500 ppm
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:06.253065 23519 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:06.253978 23519 webserver.cc:460] Webserver started at http://127.17.53.3:45683/ using document root <none> and password file <none>
I20260807 02:27:06.254537 23519 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:06.254599 23519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:06.254830 23519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:06.257050 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "b92e94194a474bf5ac0f4edc43cf94e9"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "ea30b0e93c5777384b316bf61b0675ef"
server_key_iv: "1c4ac72c15aad7342901595a999c9ec0"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.257597 23519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "b92e94194a474bf5ac0f4edc43cf94e9"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "ea30b0e93c5777384b316bf61b0675ef"
server_key_iv: "1c4ac72c15aad7342901595a999c9ec0"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.261478 23519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:06.264078 23544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.265458 23519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 02:27:06.265617 23519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "b92e94194a474bf5ac0f4edc43cf94e9"
format_stamp: "Formatted at 2026-08-07 02:27:06 on dist-test-slave-g170"
server_key: "ea30b0e93c5777384b316bf61b0675ef"
server_key_iv: "1c4ac72c15aad7342901595a999c9ec0"
server_key_version: "encryptionkey@0"
I20260807 02:27:06.265784 23519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:06.286064 23519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:06.289222 23519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:06.289426 23519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:06.290051 23519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:06.290947 23519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:06.291016 23519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.291087 23519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:06.291134 23519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:06.300839 23519 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38151
I20260807 02:27:06.300890 23717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38151 every 8 connection(s)
I20260807 02:27:06.301807 23519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:06.304471 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 23519
I20260807 02:27:06.304626 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:06.305047 17620 external_mini_cluster.cc:1428] Setting key c01a9ac3167d5d12611b41dc312c5fc5
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:06.316936 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.303877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:41275 (local address 127.17.53.62:38473)
0807 02:27:06.304021 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:06.304025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:06.305007 (+ 982us) server_negotiation.cc:407] Connection header received
0807 02:27:06.306723 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:06.306726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:06.306791 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:06.306877 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:06.308332 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.308925 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.310257 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.310484 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.310587 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.313345 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:06.313347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:06.313350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:06.313374 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:06.314874 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.315388 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.315390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.315392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.315467 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.315847 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.315850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.315851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.316027 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:06.316103 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:06.316631 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 02:27:06.316746 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:27:06.317847 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.304255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38473 (local address 127.17.53.3:41275)
0807 02:27:06.304845 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:06.304901 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:06.306502 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:06.307063 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:06.307072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:06.307592 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:06.308156 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:06.308171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.309053 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.309057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:06.310109 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:06.310121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.310642 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.310645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:06.310859 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.311436 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:06.311453 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:06.313166 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:06.314989 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:06.314994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:06.315008 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:06.315295 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:06.315557 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:06.315560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:06.315561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:06.315753 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:06.316121 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:06.316126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:06.316373 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 02:27:06.316543 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 02:27:06.316930 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":165,"threads_started":1}
I20260807 02:27:06.319082 23718 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38473
I20260807 02:27:06.319382 23718 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:06.319918 23718 heartbeater.cc:511] Master 127.17.53.62:38473 requested a full tablet report, sending...
I20260807 02:27:06.321023 23017 ts_manager.cc:194] Registered new tserver with Master: b92e94194a474bf5ac0f4edc43cf94e9 (127.17.53.3:38151)
I20260807 02:27:06.321565 23017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:41275
I20260807 02:27:06.331423 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:27:06.360060 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.350482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.17.53.1:39587)
0807 02:27:06.350835 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:06.350852 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:06.350897 (+ 45us) server_negotiation.cc:407] Connection header received
0807 02:27:06.350951 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:06.350956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:06.351157 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:06.351282 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:06.352036 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.352919 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.353736 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.354116 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.354214 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.356591 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:06.356600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:06.356613 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:06.356636 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:06.358252 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.358699 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.358704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.358706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.358791 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.359066 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.359069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.359070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.359363 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:06.359533 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:06.359705 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 02:27:06.359884 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":91,"threads_started":1}
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[22959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:06.374900 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:06.362872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.17.53.62:38473)
0807 02:27:06.363030 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:06.363033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:06.363048 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:06.363142 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:06.363144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:06.363185 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:06.363251 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:06.363856 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.364366 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.365132 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:06.365343 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:06.365419 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:06.367651 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:06.367654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:06.367657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:06.367681 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:06.373250 (+ 5569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.373654 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.373656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.373658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.373758 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:06.374057 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:06.374060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:06.374061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:06.374329 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:06.374449 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:06.374668 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 02:27:06.374779 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 02:27:06.375329 23017 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32816
I20260807 02:27:06.375985 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 23127
I20260807 02:27:06.396368 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 23315
I20260807 02:27:06.412765 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 23519
I20260807 02:27:06.423590 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 22978
2026-08-07T02:27:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2499 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[23736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[23736](info): set up 2 sockets
Aug 07 02:27:06 dist-test-slave-g170 krb5kdc[23736](info): commencing operation
krb5kdc: starting...
W20260807 02:27:07.778620 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:07 dist-test-slave-g170 krb5kdc[23736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:07.920250 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37977
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:37977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:08.040964 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:08.041285 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:08.041363 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:08.045416 23756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:08.045506 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:08.045545 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:08.045598 23756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:08.045652 23756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:08.051123 23756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:37977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:08.052204 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:08.053267 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:08.061610 23764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.061606 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.061614 23771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.062058 23756 server_base.cc:1034] running on GCE node
I20260807 02:27:08.062613 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:08.063704 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:08.064867 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786069628064840 us; error 32 us; skew 500 ppm
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:08.067938 23756 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:08.068655 23756 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:08.068720 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:08.068965 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:08.071046 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "52b6b2f97c2847cb8b36379bb9317ccd"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "06933fa6c241f284ecfa89f8a9d169cc"
server_key_iv: "5efd966b359acd404973bd191d28a876"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.071580 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "52b6b2f97c2847cb8b36379bb9317ccd"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "06933fa6c241f284ecfa89f8a9d169cc"
server_key_iv: "5efd966b359acd404973bd191d28a876"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.075471 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:27:08.078377 23781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.079768 23756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:27:08.079896 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "52b6b2f97c2847cb8b36379bb9317ccd"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "06933fa6c241f284ecfa89f8a9d169cc"
server_key_iv: "5efd966b359acd404973bd191d28a876"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.079989 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:08.091593 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:08.094738 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:08.094899 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:08.102182 23756 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:37977
I20260807 02:27:08.102500 23854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:37977 every 8 connection(s)
I20260807 02:27:08.102895 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:08.105914 23856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:08.108356 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 23756
I20260807 02:27:08.108492 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:08.108811 17620 external_mini_cluster.cc:1428] Setting key 2cb9158ce86bd8aec6d0a3d283fb43e6
I20260807 02:27:08.112224 23856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: Bootstrap starting.
I20260807 02:27:08.114866 23856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:08.115746 23856 log.cc:824] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:08.118196 23856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: No bootstrap required, opened a new log
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:08.123248 23856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } }
I20260807 02:27:08.123531 23856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:08.123601 23856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b6b2f97c2847cb8b36379bb9317ccd, State: Initialized, Role: FOLLOWER
I20260807 02:27:08.124105 23856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } }
I20260807 02:27:08.124274 23856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:08.124336 23856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:08.124430 23856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:08.124653 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.110349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.17.53.62:37977)
0807 02:27:08.110807 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.110825 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.110863 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:27:08.111553 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.111571 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.111924 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.112232 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:08.113132 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.114325 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.115520 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.116107 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.116250 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.119155 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:08.119161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:08.119178 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:08.119210 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:08.121322 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.121788 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.121794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.121802 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.121971 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.122247 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.122251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.122253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.122823 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:08.123039 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.123318 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.123640 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":149,"threads_started":1}
I20260807 02:27:08.125646 23856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } }
I20260807 02:27:08.126025 23856 leader_election.cc:302] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b6b2f97c2847cb8b36379bb9317ccd; no voters:
I20260807 02:27:08.126338 23856 leader_election.cc:288] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:08.126444 23864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:08.126665 23864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [term 1 LEADER]: Becoming Leader. State: Replica: 52b6b2f97c2847cb8b36379bb9317ccd, State: Running, Role: LEADER
I20260807 02:27:08.127065 23864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } }
I20260807 02:27:08.127655 23856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:08.128800 23864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b6b2f97c2847cb8b36379bb9317ccd. Latest consensus state: current_term: 1 leader_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } } }
I20260807 02:27:08.129019 23864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:08.129236 23865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b6b2f97c2847cb8b36379bb9317ccd" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37977 } } }
I20260807 02:27:08.129420 23865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:08.129565 23876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:08.132802 23876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:08.138106 23876 catalog_manager.cc:1347] Generated new cluster ID: e22cfcb42c6b4a3bbb4718456644b875
I20260807 02:27:08.138177 23876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:08.148504 23876 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:08.149977 23876 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:08.160925 23876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: Generated new TSK 0
I20260807 02:27:08.161835 23876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:08.196552 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:08.317342 23896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:08.317612 23896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:08.317657 23896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:08.321583 23896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:08.321636 23896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:08.321695 23896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:08.326890 23896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:08.328069 23896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:08.329218 23896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:08.338964 23903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.338974 23910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.338958 23904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.339160 23896 server_base.cc:1034] running on GCE node
I20260807 02:27:08.339722 23896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:08.340413 23896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:08.341611 23896 hybrid_clock.cc:648] HybridClock initialized: now 1786069628341593 us; error 29 us; skew 500 ppm
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:08.344960 23896 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:08.345753 23896 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:08.345849 23896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:08.346174 23896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:08.349232 23896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "df4e063f9a2547cf97bca1d845194d31"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "94cc43ea96da20688841a6de0e08fb29"
server_key_iv: "5e6cb7be3871a84388058cc2f51793d2"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.349852 23896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "df4e063f9a2547cf97bca1d845194d31"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "94cc43ea96da20688841a6de0e08fb29"
server_key_iv: "5e6cb7be3871a84388058cc2f51793d2"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.354094 23896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:08.356757 23916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.358184 23896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 02:27:08.358434 23896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "df4e063f9a2547cf97bca1d845194d31"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "94cc43ea96da20688841a6de0e08fb29"
server_key_iv: "5e6cb7be3871a84388058cc2f51793d2"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.358551 23896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:08.380465 23896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:08.383698 23896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:08.383916 23896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:08.384603 23896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:08.385512 23896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:08.385560 23896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.385622 23896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:08.385654 23896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.394970 23896 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:38609
I20260807 02:27:08.395081 24082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:38609 every 8 connection(s)
I20260807 02:27:08.395627 23896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:08.404884 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 23896
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:08.405017 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:08.405290 17620 external_mini_cluster.cc:1428] Setting key bee669c0bcf00a42a26b8cf42422d103
I20260807 02:27:08.409731 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.397445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:37635 (local address 127.17.53.62:37977)
0807 02:27:08.397591 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.397595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.398770 (+ 1175us) server_negotiation.cc:407] Connection header received
0807 02:27:08.399663 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.399667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.399723 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.399800 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:08.401011 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.401553 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.402698 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.402930 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.403035 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.405585 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:08.405589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:08.405591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:08.405619 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:08.407335 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.407903 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.407910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.407914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.408013 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.408420 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.408426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.408431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.408700 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:08.408841 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.409399 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.409541 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:08.410633 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.397813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37977 (local address 127.17.53.1:37635)
0807 02:27:08.398618 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.398659 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.399462 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.399963 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.399970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.400333 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:08.400869 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.400885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.401682 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.401686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.402566 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.402575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.403085 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.403089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.403263 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.403828 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:08.403845 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:08.405415 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:08.407461 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.407466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.407477 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.407799 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.408143 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.408147 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.408148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.408309 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.408865 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:08.408871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:08.409118 (+ 247us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.409332 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.409699 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":112,"threads_started":1}
I20260807 02:27:08.412024 24083 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37977
I20260807 02:27:08.412312 24083 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:08.412891 24083 heartbeater.cc:511] Master 127.17.53.62:37977 requested a full tablet report, sending...
I20260807 02:27:08.414779 23801 ts_manager.cc:194] Registered new tserver with Master: df4e063f9a2547cf97bca1d845194d31 (127.17.53.1:38609)
I20260807 02:27:08.416515 23801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:37635
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:08.436131 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:08.549304 24089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:08.549525 24089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:08.549599 24089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:08.553586 24089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:08.553650 24089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:08.553761 24089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:08.559254 24089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:08.560256 24089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:08.561307 24089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:08.570694 24094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.570901 24089 server_base.cc:1034] running on GCE node
W20260807 02:27:08.570706 24097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.570694 24095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.571430 24089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:08.572088 24089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:08.573268 24089 hybrid_clock.cc:648] HybridClock initialized: now 1786069628573246 us; error 40 us; skew 500 ppm
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:08.575989 24089 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:08.576686 24089 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:08.576798 24089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:08.577035 24089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:08.579047 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "6cc484d96fe5449e9ccffb3d1bd992bc"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "22453bf13ad3245adf84547994ee184f"
server_key_iv: "5dbfbd10d1aedda005a2676cd3be5d0a"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.579613 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "6cc484d96fe5449e9ccffb3d1bd992bc"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "22453bf13ad3245adf84547994ee184f"
server_key_iv: "5dbfbd10d1aedda005a2676cd3be5d0a"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.583281 24089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 02:27:08.585889 24105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.587073 24089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:08.587204 24089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "6cc484d96fe5449e9ccffb3d1bd992bc"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "22453bf13ad3245adf84547994ee184f"
server_key_iv: "5dbfbd10d1aedda005a2676cd3be5d0a"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.587320 24089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:08.598629 24089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:08.601730 24089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:08.601912 24089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:08.602500 24089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:08.603305 24089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:08.603374 24089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.603438 24089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:08.603484 24089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.612032 24089 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:35591
I20260807 02:27:08.612095 24271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:35591 every 8 connection(s)
I20260807 02:27:08.612673 24089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:08.622709 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 24089
I20260807 02:27:08.622859 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:08.623238 17620 external_mini_cluster.cc:1428] Setting key 086f11db10f90e70f5ae7e53bec43265
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:08.630164 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.614352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:38465 (local address 127.17.53.62:37977)
0807 02:27:08.614497 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.614501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.615672 (+ 1171us) server_negotiation.cc:407] Connection header received
0807 02:27:08.616487 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.616490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.616538 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.616738 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:08.617952 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.618493 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.620389 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.620849 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.620963 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.625795 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:08.625798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:08.625801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:08.625830 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:08.627433 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.628015 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.628020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.628023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.628140 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.628638 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.628643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.628646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.628928 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:08.629010 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.629758 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.629913 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:27:08.631982 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.614745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37977 (local address 127.17.53.2:38465)
0807 02:27:08.615522 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.615561 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.616295 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.616933 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.616940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.617318 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:08.617788 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.617802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.618634 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.618638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.620247 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.620262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.621182 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.621186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.621415 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.622141 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:08.622170 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:08.625631 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:08.627564 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.627571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.627585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.627858 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.628277 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.628282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.628285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.628469 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.629102 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:08.629109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:08.629449 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.630412 (+ 963us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.630799 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260807 02:27:08.633374 24273 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37977
I20260807 02:27:08.633654 24273 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:08.634234 24273 heartbeater.cc:511] Master 127.17.53.62:37977 requested a full tablet report, sending...
I20260807 02:27:08.635556 23801 ts_manager.cc:194] Registered new tserver with Master: 6cc484d96fe5449e9ccffb3d1bd992bc (127.17.53.2:35591)
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:27:08.636485 23801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:38465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:08.655966 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:08.775745 24280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:08.776057 24280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:08.776145 24280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:08.780423 24280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:08.780531 24280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:08.780632 24280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:08.786476 24280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:08.787628 24280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:08.788700 24280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:08.799055 24285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.799325 24280 server_base.cc:1034] running on GCE node
W20260807 02:27:08.799055 24286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:08.799111 24289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:08.799925 24280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:08.800622 24280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:08.801820 24280 hybrid_clock.cc:648] HybridClock initialized: now 1786069628801788 us; error 53 us; skew 500 ppm
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:08.804752 24280 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:08.805598 24280 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:08.805689 24280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:08.805913 24280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:08.808032 24280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "d8c30f8730534155af1ae1ebf957b510"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "f0e285f0f64c2fddfa7e3e4b20f8e87e"
server_key_iv: "00a05b320f1781bcaf05a60bfb3ebe56"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.808648 24280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "d8c30f8730534155af1ae1ebf957b510"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "f0e285f0f64c2fddfa7e3e4b20f8e87e"
server_key_iv: "00a05b320f1781bcaf05a60bfb3ebe56"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.812564 24280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 02:27:08.815249 24296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.816618 24280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:08.816785 24280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "d8c30f8730534155af1ae1ebf957b510"
format_stamp: "Formatted at 2026-08-07 02:27:08 on dist-test-slave-g170"
server_key: "f0e285f0f64c2fddfa7e3e4b20f8e87e"
server_key_iv: "00a05b320f1781bcaf05a60bfb3ebe56"
server_key_version: "encryptionkey@0"
I20260807 02:27:08.816924 24280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:08.833877 24280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:08.837227 24280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:08.837435 24280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:08.838032 24280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:08.838901 24280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:08.838970 24280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.839039 24280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:08.839085 24280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:08.848276 24280 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:42811
I20260807 02:27:08.848290 24456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:42811 every 8 connection(s)
I20260807 02:27:08.849037 24280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:08.853680 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 24280
I20260807 02:27:08.853829 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:08.854120 17620 external_mini_cluster.cc:1428] Setting key dac8afdadc6605f7d05414610ad2c254
Aug 07 02:27:08 dist-test-slave-g170 krb5kdc[23736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069628, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:08.863633 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.850914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:33313 (local address 127.17.53.62:37977)
0807 02:27:08.851079 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.851082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.852459 (+ 1377us) server_negotiation.cc:407] Connection header received
0807 02:27:08.853493 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.853497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.853557 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.853664 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:08.855070 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.855617 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.856894 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.857111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.857221 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.860002 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:08.860006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:08.860008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:08.860031 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:08.861368 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.861918 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.861924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.861929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.862011 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.862415 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.862419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.862421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.862712 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:08.862822 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.863340 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.863464 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 02:27:08.864387 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.851273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:37977 (local address 127.17.53.3:33313)
0807 02:27:08.852206 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.852249 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.853268 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.853866 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.853876 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.854355 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:08.854915 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.854940 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.855745 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.855749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.856731 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.856741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.857263 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.857267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.857442 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.858075 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:08.858092 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:08.859832 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:08.861489 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.861495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.861509 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.861816 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.862104 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:08.862108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:08.862110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:08.862322 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:08.862821 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:08.862828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:08.863070 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.863264 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.863584 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":120,"threads_started":1}
I20260807 02:27:08.865527 24457 heartbeater.cc:347] Connected to a master server at 127.17.53.62:37977
I20260807 02:27:08.865797 24457 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:08.866212 24457 heartbeater.cc:511] Master 127.17.53.62:37977 requested a full tablet report, sending...
I20260807 02:27:08.867439 23801 ts_manager.cc:194] Registered new tserver with Master: d8c30f8730534155af1ae1ebf957b510 (127.17.53.3:42811)
I20260807 02:27:08.868036 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:27:08.868067 23801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:33313
I20260807 02:27:08.877640 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.870214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.17.53.62:37977)
0807 02:27:08.870347 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.870352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.870515 (+ 163us) server_negotiation.cc:407] Connection header received
0807 02:27:08.870608 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.870610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.870645 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.870708 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:08.871446 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.871935 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.872885 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.873088 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.873167 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.874475 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:08.874478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:08.874480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:08.874501 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:08.875732 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.876189 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.876193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.876194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.876279 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:08.876586 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:08.876588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:08.876589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:08.876936 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:08.877099 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.877333 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.877451 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:08.880394 23801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:08.883461 23801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:08.898455 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.892672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:35591 (local address 127.0.0.1:39550)
0807 02:27:08.893189 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.893206 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.893300 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.893963 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.893966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.893979 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:08.894048 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.894052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895736 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.895739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.897038 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.897050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.897889 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.897893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.897999 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898037 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.898147 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.898284 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":68,"thread_start_us":129,"threads_started":1}
I20260807 02:27:08.898463 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.893115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.17.53.2:35591)
0807 02:27:08.893543 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.893552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.893571 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:27:08.893635 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.893639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.893813 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.894218 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:08.894381 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895560 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.897199 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.897752 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.897890 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898051 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.898190 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.898288 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":125,"threads_started":1}
I20260807 02:27:08.898854 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.891958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38609 (local address 127.0.0.1:48254)
0807 02:27:08.892326 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.892385 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.893270 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.893853 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.893862 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.893891 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:08.894045 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.894051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895140 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.895147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.897038 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.897050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.898251 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.898254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.898485 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898537 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.898606 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.898686 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":85,"thread_start_us":86,"threads_started":1}
I20260807 02:27:08.899241 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.892257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42811 (local address 127.0.0.1:39150)
0807 02:27:08.892636 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:08.892663 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:08.892841 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:08.894141 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:08.894145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:08.894165 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:08.894282 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.894289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895737 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.895744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.896951 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:08.896961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.898251 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.898254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:08.898479 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898532 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 02:27:08.899016 (+ 484us) client_negotiation.cc:240] Negotiation successful
0807 02:27:08.899104 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":115,"thread_start_us":116,"threads_started":1}
I20260807 02:27:08.899652 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.892742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39150 (local address 127.17.53.3:42811)
0807 02:27:08.893554 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.893565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.893586 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:08.893659 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.893664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.893836 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.893975 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:08.894412 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895560 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.897074 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.897915 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.898033 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898181 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.899410 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.899512 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":105,"threads_started":1}
I20260807 02:27:08.899660 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:08.892068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.17.53.1:38609)
0807 02:27:08.892456 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:08.892464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:08.892489 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:27:08.893438 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:08.893446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:08.893665 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:08.893839 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:08.894171 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.895012 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.897152 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:08.897936 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:08.898033 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:08.898175 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:08.899194 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0807 02:27:08.899490 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":106,"threads_started":1}
I20260807 02:27:08.902177 24174 tablet_service.cc:1467] Processing CreateTablet for tablet ef699741bd2e4686b561a4de70ace093 (DEFAULT_TABLE table=test-table [id=e6a613ae06664848ba3fc03198a1fbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:08.902171 23981 tablet_service.cc:1467] Processing CreateTablet for tablet ef699741bd2e4686b561a4de70ace093 (DEFAULT_TABLE table=test-table [id=e6a613ae06664848ba3fc03198a1fbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:08.903402 23981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef699741bd2e4686b561a4de70ace093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:08.903402 24174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef699741bd2e4686b561a4de70ace093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:08.903016 24363 tablet_service.cc:1467] Processing CreateTablet for tablet ef699741bd2e4686b561a4de70ace093 (DEFAULT_TABLE table=test-table [id=e6a613ae06664848ba3fc03198a1fbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:08.904007 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef699741bd2e4686b561a4de70ace093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:08.910032 24484 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Bootstrap starting.
I20260807 02:27:08.911607 24488 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Bootstrap starting.
I20260807 02:27:08.912467 24484 tablet_bootstrap.cc:654] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:08.913089 24489 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Bootstrap starting.
I20260807 02:27:08.913550 24488 tablet_bootstrap.cc:654] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:08.913691 24484 log.cc:824] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:08.914743 24488 log.cc:824] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:08.915333 24489 tablet_bootstrap.cc:654] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:08.915598 24484 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: No bootstrap required, opened a new log
I20260807 02:27:08.915812 24484 ts_tablet_manager.cc:1397] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 02:27:08.916126 24489 log.cc:824] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:08.916849 24488 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: No bootstrap required, opened a new log
I20260807 02:27:08.917049 24488 ts_tablet_manager.cc:1397] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:27:08.918049 24489 tablet_bootstrap.cc:492] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: No bootstrap required, opened a new log
I20260807 02:27:08.918223 24489 ts_tablet_manager.cc:1397] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:27:08.919059 24484 raft_consensus.cc:348] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.919301 24484 raft_consensus.cc:374] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:08.919385 24484 raft_consensus.cc:729] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c30f8730534155af1ae1ebf957b510, State: Initialized, Role: FOLLOWER
I20260807 02:27:08.919858 24484 consensus_queue.cc:260] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.921016 24484 ts_tablet_manager.cc:1428] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260807 02:27:08.920967 24488 raft_consensus.cc:348] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.921226 24488 raft_consensus.cc:374] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:08.921283 24488 raft_consensus.cc:729] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc484d96fe5449e9ccffb3d1bd992bc, State: Initialized, Role: FOLLOWER
I20260807 02:27:08.921402 24457 heartbeater.cc:503] Master 127.17.53.62:37977 was elected leader, sending a full tablet report...
I20260807 02:27:08.921738 24488 consensus_queue.cc:260] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.922391 24273 heartbeater.cc:503] Master 127.17.53.62:37977 was elected leader, sending a full tablet report...
I20260807 02:27:08.922520 24489 raft_consensus.cc:348] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.922757 24489 raft_consensus.cc:374] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:08.922875 24489 raft_consensus.cc:729] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4e063f9a2547cf97bca1d845194d31, State: Initialized, Role: FOLLOWER
I20260807 02:27:08.923262 24488 ts_tablet_manager.cc:1428] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:27:08.923413 24489 consensus_queue.cc:260] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:08.924091 24083 heartbeater.cc:503] Master 127.17.53.62:37977 was elected leader, sending a full tablet report...
I20260807 02:27:08.925343 24489 ts_tablet_manager.cc:1428] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260807 02:27:09.099985 24458 tablet.cc:2369] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:09.105767 24495 raft_consensus.cc:482] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:09.105997 24495 raft_consensus.cc:504] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:09.107125 24495 leader_election.cc:288] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df4e063f9a2547cf97bca1d845194d31 (127.17.53.1:38609), 6cc484d96fe5449e9ccffb3d1bd992bc (127.17.53.2:35591)
I20260807 02:27:09.111800 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.107739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:35591 (local address 127.17.53.3:44287)
0807 02:27:09.108108 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.108121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:09.108231 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:09.108486 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:09.108489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:09.108506 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:09.108643 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:09.108649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.109668 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.109672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:09.110612 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:09.110623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.111310 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.111314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:09.111424 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:09.111453 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 02:27:09.111528 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 02:27:09.111609 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":55,"thread_start_us":136,"threads_started":1}
I20260807 02:27:09.111795 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.107965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:44287 (local address 127.17.53.2:35591)
0807 02:27:09.108103 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.108107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.108203 (+ 96us) server_negotiation.cc:407] Connection header received
0807 02:27:09.108322 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.108326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.108380 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.108459 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:09.108780 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.109509 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.110741 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.111186 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.111297 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:09.111368 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.111575 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.111656 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 02:27:09.112455 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.107471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38609 (local address 127.17.53.3:35569)
0807 02:27:09.107748 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.107766 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:09.107902 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:09.108385 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:09.108389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:09.108410 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:09.108546 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:09.108553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.109656 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.109660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:09.111054 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:09.111077 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.112043 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.112047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:09.112151 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:09.112173 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 02:27:09.112254 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 02:27:09.112334 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":3}
I20260807 02:27:09.112499 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.107578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:35569 (local address 127.17.53.1:38609)
0807 02:27:09.107714 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.107719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.108007 (+ 288us) server_negotiation.cc:407] Connection header received
0807 02:27:09.108074 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.108077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.108139 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.108237 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:09.108671 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.109510 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.111222 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.111800 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.111908 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:09.111978 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.112280 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.112372 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:27:09.112674 24205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef699741bd2e4686b561a4de70ace093" candidate_uuid: "d8c30f8730534155af1ae1ebf957b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" is_pre_election: true
I20260807 02:27:09.113097 24205 raft_consensus.cc:2454] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c30f8730534155af1ae1ebf957b510 in term 0.
I20260807 02:27:09.113154 24016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef699741bd2e4686b561a4de70ace093" candidate_uuid: "d8c30f8730534155af1ae1ebf957b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e063f9a2547cf97bca1d845194d31" is_pre_election: true
I20260807 02:27:09.113454 24016 raft_consensus.cc:2454] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c30f8730534155af1ae1ebf957b510 in term 0.
I20260807 02:27:09.113615 24297 leader_election.cc:302] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc484d96fe5449e9ccffb3d1bd992bc, d8c30f8730534155af1ae1ebf957b510; no voters:
W20260807 02:27:09.113886 24274 tablet.cc:2369] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:09.113924 24495 raft_consensus.cc:2781] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:09.114005 24495 raft_consensus.cc:482] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:09.114048 24495 raft_consensus.cc:3037] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:09.115110 24495 raft_consensus.cc:504] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:09.115496 24495 leader_election.cc:288] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [CANDIDATE]: Term 1 election: Requested vote from peers df4e063f9a2547cf97bca1d845194d31 (127.17.53.1:38609), 6cc484d96fe5449e9ccffb3d1bd992bc (127.17.53.2:35591)
I20260807 02:27:09.115880 24016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef699741bd2e4686b561a4de70ace093" candidate_uuid: "d8c30f8730534155af1ae1ebf957b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4e063f9a2547cf97bca1d845194d31"
I20260807 02:27:09.115931 24205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef699741bd2e4686b561a4de70ace093" candidate_uuid: "d8c30f8730534155af1ae1ebf957b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc"
I20260807 02:27:09.116024 24016 raft_consensus.cc:3037] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:09.116057 24205 raft_consensus.cc:3037] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:09.117121 24016 raft_consensus.cc:2454] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c30f8730534155af1ae1ebf957b510 in term 1.
I20260807 02:27:09.117422 24297 leader_election.cc:302] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8c30f8730534155af1ae1ebf957b510, df4e063f9a2547cf97bca1d845194d31; no voters:
I20260807 02:27:09.117514 24205 raft_consensus.cc:2454] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c30f8730534155af1ae1ebf957b510 in term 1.
I20260807 02:27:09.117604 24495 raft_consensus.cc:2781] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:09.117846 24495 raft_consensus.cc:686] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 1 LEADER]: Becoming Leader. State: Replica: d8c30f8730534155af1ae1ebf957b510, State: Running, Role: LEADER
I20260807 02:27:09.118196 24495 consensus_queue.cc:237] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } }
I20260807 02:27:09.121800 23801 catalog_manager.cc:5495] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8c30f8730534155af1ae1ebf957b510 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "d8c30f8730534155af1ae1ebf957b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c30f8730534155af1ae1ebf957b510" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42811 } health_report { overall_health: HEALTHY } } }
I20260807 02:27:09.132472 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.129057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39156 (local address 127.17.53.3:42811)
0807 02:27:09.129177 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.129180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.129216 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:27:09.129354 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.129356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.129409 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.129497 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:09.129711 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.130273 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.131115 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:09.131318 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:09.131393 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:09.131654 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:09.132076 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:09.132174 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.132228 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.132277 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 02:27:09.138456 24016 raft_consensus.cc:1261] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31 [term 1 FOLLOWER]: Refusing update from remote peer d8c30f8730534155af1ae1ebf957b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:09.138456 24205 raft_consensus.cc:1261] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 1 FOLLOWER]: Refusing update from remote peer d8c30f8730534155af1ae1ebf957b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:09.139333 24503 consensus_queue.cc:1035] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4e063f9a2547cf97bca1d845194d31" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:09.139714 24495 consensus_queue.cc:1035] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc484d96fe5449e9ccffb3d1bd992bc" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 02:27:09.146922 24085 tablet.cc:2369] T ef699741bd2e4686b561a4de70ace093 P df4e063f9a2547cf97bca1d845194d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:09.148512 24508 mvcc.cc:205] Tried to move back new op lower bound from 7315741200942931968 to 7315741200870674432. Current Snapshot: MvccSnapshot[applied={T|T < 7315741200942931968}]
I20260807 02:27:09.151214 24507 mvcc.cc:205] Tried to move back new op lower bound from 7315741200942931968 to 7315741200870674432. Current Snapshot: MvccSnapshot[applied={T|T < 7315741200942931968}]
I20260807 02:27:09.157316 23801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:09.157475 23801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:09.160187 23801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52b6b2f97c2847cb8b36379bb9317ccd: Sending DeleteTablet for 3 replicas of tablet ef699741bd2e4686b561a4de70ace093
I20260807 02:27:09.161118 24363 tablet_service.cc:1514] Processing DeleteTablet for tablet ef699741bd2e4686b561a4de70ace093 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:09 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:39150
I20260807 02:27:09.161319 23981 tablet_service.cc:1514] Processing DeleteTablet for tablet ef699741bd2e4686b561a4de70ace093 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:09 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48254
I20260807 02:27:09.161757 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 23896
I20260807 02:27:09.161870 24174 tablet_service.cc:1514] Processing DeleteTablet for tablet ef699741bd2e4686b561a4de70ace093 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:09 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:39550
I20260807 02:27:09.169545 24521 tablet_replica.cc:330] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: stopping tablet replica
I20260807 02:27:09.170001 24521 raft_consensus.cc:2229] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:09.170400 24521 raft_consensus.cc:2258] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:09.170857 24519 tablet_replica.cc:330] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: stopping tablet replica
I20260807 02:27:09.171244 24519 raft_consensus.cc:2229] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:09.171648 24519 raft_consensus.cc:2258] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:09.175695 24519 ts_tablet_manager.cc:1905] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:09.178004 24521 ts_tablet_manager.cc:1905] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:09.179324 24519 ts_tablet_manager.cc:1918] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:09.179548 24519 log.cc:1196] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/ef699741bd2e4686b561a4de70ace093
I20260807 02:27:09.180073 24519 ts_tablet_manager.cc:1939] T ef699741bd2e4686b561a4de70ace093 P d8c30f8730534155af1ae1ebf957b510: Deleting consensus metadata
I20260807 02:27:09.181281 24521 ts_tablet_manager.cc:1918] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:09.181492 24521 log.cc:1196] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/ef699741bd2e4686b561a4de70ace093
I20260807 02:27:09.181850 23785 catalog_manager.cc:4841] TS d8c30f8730534155af1ae1ebf957b510 (127.17.53.3:42811): tablet ef699741bd2e4686b561a4de70ace093 (table test-table [id=e6a613ae06664848ba3fc03198a1fbf0]) successfully deleted
I20260807 02:27:09.181942 24521 ts_tablet_manager.cc:1939] T ef699741bd2e4686b561a4de70ace093 P 6cc484d96fe5449e9ccffb3d1bd992bc: Deleting consensus metadata
W20260807 02:27:09.182667 23782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:38609 (error 108)
I20260807 02:27:09.182912 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 24089
I20260807 02:27:09.189427 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.188988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38609 (local address 127.0.0.1:48264)
0807 02:27:09.189191 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.189292 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 02:27:09.189659 23782 catalog_manager.cc:4584] TS df4e063f9a2547cf97bca1d845194d31 (127.17.53.1:38609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef699741bd2e4686b561a4de70ace093: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38609: connect: Connection refused (error 111)
I20260807 02:27:09.195472 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 24280
I20260807 02:27:09.195767 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.195395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:35591 (local address 127.0.0.1:39564)
0807 02:27:09.195602 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.195656 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.2:35591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 02:27:09.214051 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 23756
2026-08-07T02:27:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2789 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T02:27:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:09Z Disabled control of system clock
I20260807 02:27:09.254045 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34213
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:34213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 02:27:09.375589 24527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:09.375854 24527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:09.375895 24527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:09.379912 24527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:09.379971 24527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:09.379987 24527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:09.380002 24527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:09.380017 24527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:09.384733 24527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:34213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:09.385823 24527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:09.387594 24527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:27:09.396389 24527 server_base.cc:1034] running on GCE node
W20260807 02:27:09.396399 24533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.396406 24536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.396543 24534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:09.397235 24527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:09.398301 24527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:09.399521 24527 hybrid_clock.cc:648] HybridClock initialized: now 1786069629399498 us; error 35 us; skew 500 ppm
I20260807 02:27:09.401639 24527 webserver.cc:460] Webserver started at http://127.17.53.62:42863/ using document root <none> and password file <none>
I20260807 02:27:09.402249 24527 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:09.402351 24527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:09.402590 24527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:09.404757 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "020292a155a14a4fb971fa5ee5c47a3f"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "d5800c72558b81d95a40c98ffac2e9bc"
server_key_iv: "3670f6bf19d559f8c451ecb0e4036408"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.405378 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "020292a155a14a4fb971fa5ee5c47a3f"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "d5800c72558b81d95a40c98ffac2e9bc"
server_key_iv: "3670f6bf19d559f8c451ecb0e4036408"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.409189 24527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:27:09.411616 24549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.412954 24527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:09.413107 24527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "020292a155a14a4fb971fa5ee5c47a3f"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "d5800c72558b81d95a40c98ffac2e9bc"
server_key_iv: "3670f6bf19d559f8c451ecb0e4036408"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.413226 24527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:09.417877 24527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:09.418514 24527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:09.418836 24527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:09.426872 24527 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:34213
I20260807 02:27:09.426887 24625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:34213 every 8 connection(s)
I20260807 02:27:09.427908 24527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:09.430778 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 24527
I20260807 02:27:09.430888 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:09.431216 17620 external_mini_cluster.cc:1428] Setting key ffaa26587fa1abf3706ae3a5d0e8c396
I20260807 02:27:09.431618 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:09.437693 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.432625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.17.53.62:34213)
0807 02:27:09.433423 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.433446 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.433494 (+ 48us) server_negotiation.cc:407] Connection header received
0807 02:27:09.434350 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.434376 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.434873 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.435273 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:09.436049 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:09.436055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:09.436072 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:09.436084 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:09.436156 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:09.436193 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:09.436356 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.436431 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.436699 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":180,"threads_started":1}
I20260807 02:27:09.439836 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: Bootstrap starting.
I20260807 02:27:09.442324 24627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:09.443056 24627 log.cc:824] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:09.445115 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: No bootstrap required, opened a new log
I20260807 02:27:09.448729 24627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } }
I20260807 02:27:09.449011 24627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:09.449098 24627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020292a155a14a4fb971fa5ee5c47a3f, State: Initialized, Role: FOLLOWER
I20260807 02:27:09.449553 24627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } }
I20260807 02:27:09.449698 24627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:09.449764 24627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:09.449867 24627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:09.450722 24627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } }
I20260807 02:27:09.451056 24627 leader_election.cc:302] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 020292a155a14a4fb971fa5ee5c47a3f; no voters:
I20260807 02:27:09.451383 24627 leader_election.cc:288] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:09.451521 24634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:09.451857 24634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [term 1 LEADER]: Becoming Leader. State: Replica: 020292a155a14a4fb971fa5ee5c47a3f, State: Running, Role: LEADER
I20260807 02:27:09.452201 24634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } }
I20260807 02:27:09.452847 24627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:09.453995 24637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "020292a155a14a4fb971fa5ee5c47a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } } }
I20260807 02:27:09.454118 24637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:09.454550 24645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:09.454514 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020292a155a14a4fb971fa5ee5c47a3f. Latest consensus state: current_term: 1 leader_uuid: "020292a155a14a4fb971fa5ee5c47a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020292a155a14a4fb971fa5ee5c47a3f" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34213 } } }
I20260807 02:27:09.455276 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:09.457782 24645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:09.463430 24645 catalog_manager.cc:1347] Generated new cluster ID: 85ba2039bf0a472c9ac115c46d822e4d
I20260807 02:27:09.463500 24645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:09.482563 24645 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:09.482949 24645 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:09.493115 24645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: Generated new TSK 0
I20260807 02:27:09.494047 24645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:09.507365 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--builtin_ntp_servers=127.17.53.20:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 02:27:09.624774 24666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:09.625043 24666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:09.625115 24666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:09.629087 24666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:09.629151 24666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:09.629242 24666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:09.633854 24666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:09.634910 24666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:09.636304 24666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:09.646042 24674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.646088 24673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.646042 24676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:09.646387 24666 server_base.cc:1034] running on GCE node
I20260807 02:27:09.646840 24666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:09.647464 24666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:09.648627 24666 hybrid_clock.cc:648] HybridClock initialized: now 1786069629648607 us; error 31 us; skew 500 ppm
I20260807 02:27:09.650707 24666 webserver.cc:460] Webserver started at http://127.17.53.1:34687/ using document root <none> and password file <none>
I20260807 02:27:09.651293 24666 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:09.651357 24666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:09.651567 24666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:09.653932 24666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "cbccb7ac2049466213932678fb064f78"
server_key_iv: "a6297b0d4c530733da1316708c186c8c"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.654563 24666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "cbccb7ac2049466213932678fb064f78"
server_key_iv: "a6297b0d4c530733da1316708c186c8c"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.658385 24666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:09.660996 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.662261 24666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:09.662398 24666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "cbccb7ac2049466213932678fb064f78"
server_key_iv: "a6297b0d4c530733da1316708c186c8c"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.662511 24666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:09.667100 24666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:09.667726 24666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:09.667919 24666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:09.668465 24666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:09.669500 24666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:09.669569 24666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.669632 24666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:09.669677 24666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.679081 24666 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:46639
I20260807 02:27:09.679150 24861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:46639 every 8 connection(s)
I20260807 02:27:09.680065 24666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:09.685108 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 24666
I20260807 02:27:09.685294 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:09.685616 17620 external_mini_cluster.cc:1428] Setting key e1e69d860a636c4839b90c52d12c6552
I20260807 02:27:09.687016 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.682031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:57397 (local address 127.17.53.62:34213)
0807 02:27:09.682182 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.682186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.683361 (+ 1175us) server_negotiation.cc:407] Connection header received
0807 02:27:09.684883 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.684888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.684962 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.685083 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:09.686426 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:09.686431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:09.686439 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:09.686453 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:09.686474 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:09.686484 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:09.686589 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.686820 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.686873 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:27:09.687640 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--builtin_ntp_servers=127.17.53.20:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 02:27:09.688234 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.682429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34213 (local address 127.17.53.1:57397)
0807 02:27:09.683118 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.683159 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:09.684574 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:09.685290 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:09.685300 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:09.685439 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:09.686128 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:09.686130 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:09.686175 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:09.686258 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:09.686574 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:09.686584 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:09.686590 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:27:09.686876 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 02:27:09.687339 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
I20260807 02:27:09.689579 24865 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34213
I20260807 02:27:09.689872 24865 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:09.690338 24865 heartbeater.cc:511] Master 127.17.53.62:34213 requested a full tablet report, sending...
I20260807 02:27:09.692158 24574 ts_manager.cc:194] Registered new tserver with Master: ffa1ce6f2aab4ee49ef367a24dc7fe5a (127.17.53.1:46639)
W20260807 02:27:09.808526 24873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:09.808876 24873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:09.808928 24873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:09.812952 24873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:09.813001 24873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:09.813056 24873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:09.817827 24873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:09.818954 24873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:09.820336 24873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:09.830199 24887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.830178 24883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:09.830231 24882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:09.830538 24873 server_base.cc:1034] running on GCE node
I20260807 02:27:09.830952 24873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:09.831647 24873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:09.832831 24873 hybrid_clock.cc:648] HybridClock initialized: now 1786069629832797 us; error 50 us; skew 500 ppm
I20260807 02:27:09.834724 24873 webserver.cc:460] Webserver started at http://127.17.53.2:45455/ using document root <none> and password file <none>
I20260807 02:27:09.835345 24873 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:09.835439 24873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:09.835667 24873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:09.837807 24873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "c22e4c20e5ce464c802fdac33db47ded"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "8611bdf0d4354fda5d61c958f4afbc63"
server_key_iv: "6b1912e1a81af4bbe373190754cab647"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.838419 24873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "c22e4c20e5ce464c802fdac33db47ded"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "8611bdf0d4354fda5d61c958f4afbc63"
server_key_iv: "6b1912e1a81af4bbe373190754cab647"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.842242 24873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:09.845120 24897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.846500 24873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:27:09.846657 24873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "c22e4c20e5ce464c802fdac33db47ded"
format_stamp: "Formatted at 2026-08-07 02:27:09 on dist-test-slave-g170"
server_key: "8611bdf0d4354fda5d61c958f4afbc63"
server_key_iv: "6b1912e1a81af4bbe373190754cab647"
server_key_version: "encryptionkey@0"
I20260807 02:27:09.846786 24873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:09.851861 24873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:09.852563 24873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:09.852871 24873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:09.853482 24873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:09.854573 24873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:09.854650 24873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.854719 24873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:09.854754 24873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:09.864252 24873 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:38991
I20260807 02:27:09.864365 25057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:38991 every 8 connection(s)
I20260807 02:27:09.865401 24873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:09.865629 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 24873
I20260807 02:27:09.865760 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:09.866096 17620 external_mini_cluster.cc:1428] Setting key ac3b97dafe1f65f0774be372de859649
I20260807 02:27:09.868683 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--builtin_ntp_servers=127.17.53.20:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 02:27:09.873090 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.867681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:54649 (local address 127.17.53.62:34213)
0807 02:27:09.867838 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:09.867844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:09.869245 (+ 1401us) server_negotiation.cc:407] Connection header received
0807 02:27:09.871198 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:09.871202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:09.871255 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:09.871357 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:09.872555 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:09.872559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:09.872561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:09.872575 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:09.872599 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:09.872608 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:09.872707 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:09.872907 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 02:27:09.872957 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:09.874039 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:09.868195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34213 (local address 127.17.53.2:54649)
0807 02:27:09.869061 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:09.869119 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:09.870982 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:09.871530 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:09.871537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:09.871628 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:09.872289 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:09.872291 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:09.872332 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:09.872404 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:09.872693 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:09.872703 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:09.872709 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:27:09.872943 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 02:27:09.873192 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":122,"threads_started":1}
I20260807 02:27:09.875370 25058 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34213
I20260807 02:27:09.875614 25058 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:09.876036 25058 heartbeater.cc:511] Master 127.17.53.62:34213 requested a full tablet report, sending...
I20260807 02:27:09.877271 24574 ts_manager.cc:194] Registered new tserver with Master: c22e4c20e5ce464c802fdac33db47ded (127.17.53.2:38991)
W20260807 02:27:09.991670 25061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:09.991919 25061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:09.991958 25061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:10.001874 25061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:10.001981 25061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:10.002044 25061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:10.007089 25061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:10.008258 25061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:10.009719 25061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:10.019870 25067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:10.019919 25061 server_base.cc:1034] running on GCE node
W20260807 02:27:10.019857 25068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:10.019862 25070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:10.020524 25061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:10.021291 25061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:10.022500 25061 hybrid_clock.cc:648] HybridClock initialized: now 1786069630022484 us; error 33 us; skew 500 ppm
I20260807 02:27:10.024385 25061 webserver.cc:460] Webserver started at http://127.17.53.3:33413/ using document root <none> and password file <none>
I20260807 02:27:10.025068 25061 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:10.025161 25061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:10.025399 25061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:10.027645 25061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "c1314bb661eb446187deac8e4ef64efc"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "cbc2711f5073951609c515f7572f5742"
server_key_iv: "70b3375a350884657feeefcb57f70ad0"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.028223 25061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "c1314bb661eb446187deac8e4ef64efc"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "cbc2711f5073951609c515f7572f5742"
server_key_iv: "70b3375a350884657feeefcb57f70ad0"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.031910 25061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 02:27:10.034528 25077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:10.035766 25061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:10.035928 25061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "c1314bb661eb446187deac8e4ef64efc"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "cbc2711f5073951609c515f7572f5742"
server_key_iv: "70b3375a350884657feeefcb57f70ad0"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.036043 25061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:10.040848 25061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:10.041507 25061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:10.041695 25061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:10.042265 25061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:10.043516 25061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:10.043588 25061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:10.043648 25061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:10.043686 25061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:10.053273 25061 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:34663
I20260807 02:27:10.053380 25242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:34663 every 8 connection(s)
I20260807 02:27:10.054369 25061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:10.056408 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25061
I20260807 02:27:10.056527 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:10.056886 17620 external_mini_cluster.cc:1428] Setting key e1e85b357a59bf3c23ef3fdd7d057d68
I20260807 02:27:10.060406 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.056236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:56329 (local address 127.17.53.62:34213)
0807 02:27:10.056393 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.056398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.057648 (+ 1250us) server_negotiation.cc:407] Connection header received
0807 02:27:10.058988 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.058991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.059026 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.059091 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.060077 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.060080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.060082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.060094 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.060112 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.060120 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.060193 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.060269 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.060302 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 02:27:10.061316 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.056734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34213 (local address 127.17.53.3:56329)
0807 02:27:10.057496 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.057548 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.058803 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.059225 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.059231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.059314 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.059816 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.059817 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.059881 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.059947 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.060180 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.060185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.060188 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.060305 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.060518 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":135,"threads_started":1}
I20260807 02:27:10.062322 25245 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34213
I20260807 02:27:10.062563 25245 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:10.062979 25245 heartbeater.cc:511] Master 127.17.53.62:34213 requested a full tablet report, sending...
I20260807 02:27:10.063990 24574 ts_manager.cc:194] Registered new tserver with Master: c1314bb661eb446187deac8e4ef64efc (127.17.53.3:34663)
I20260807 02:27:10.070513 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:27:10.074023 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.072477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.17.53.62:34213)
0807 02:27:10.072617 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.072621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.072731 (+ 110us) server_negotiation.cc:407] Connection header received
0807 02:27:10.072900 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.072902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.072933 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.072999 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.073652 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.073654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.073656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.073667 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.073685 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.073693 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.073768 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.073835 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.073866 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:27:10.077046 24574 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:10.080004 24574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:10.091508 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.17.53.3:34663)
0807 02:27:10.089477 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.089490 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.089517 (+ 27us) server_negotiation.cc:407] Connection header received
0807 02:27:10.089575 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.089579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.089743 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.090246 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.090945 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.090950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.090962 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.090971 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.091009 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.091027 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.091145 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.091300 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.091373 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1165,"thread_start_us":130,"threads_started":1}
I20260807 02:27:10.091609 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:46639 (local address 127.0.0.1:50272)
0807 02:27:10.088831 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.088879 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.088983 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.089894 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.089900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.089975 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.090646 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.090649 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.090662 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.090698 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.091350 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.091353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.091359 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.091418 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.091454 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":103,"threads_started":1}
I20260807 02:27:10.091831 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.17.53.1:46639)
0807 02:27:10.089414 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.089428 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.089453 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:27:10.089507 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.089514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.089716 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.090249 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.091129 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.091135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.091148 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.091158 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.091196 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.091233 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.091548 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.091598 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.091672 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":132,"threads_started":1}
I20260807 02:27:10.091845 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:38991 (local address 127.0.0.1:50394)
0807 02:27:10.089380 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.089395 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.089469 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.090168 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.090174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.090190 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.090830 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.090832 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.090841 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.090860 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.091619 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.091622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.091625 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.091695 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.091738 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":106,"threads_started":1}
I20260807 02:27:10.091459 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:34663 (local address 127.0.0.1:51442)
0807 02:27:10.088829 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.088878 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.088997 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.089899 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.089902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.089982 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.090585 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.090589 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.090613 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.090684 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.091119 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.091129 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.091135 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.091261 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.091331 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"spinlock_wait_cycles":35584,"thread_start_us":64,"threads_started":1}
I20260807 02:27:10.092319 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.088943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.17.53.2:38991)
0807 02:27:10.089769 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.089785 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.089809 (+ 24us) server_negotiation.cc:407] Connection header received
0807 02:27:10.089865 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.089870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.090022 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.090806 (+ 784us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.091386 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.091394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.091406 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.091416 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.091454 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.091481 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.091578 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.092093 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.092166 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":136,"threads_started":1}
I20260807 02:27:10.094653 24762 tablet_service.cc:1467] Processing CreateTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 (DEFAULT_TABLE table=test-table [id=025376c7c900425ba5db982bca51ee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:10.094996 25150 tablet_service.cc:1467] Processing CreateTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 (DEFAULT_TABLE table=test-table [id=025376c7c900425ba5db982bca51ee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:10.095000 24959 tablet_service.cc:1467] Processing CreateTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 (DEFAULT_TABLE table=test-table [id=025376c7c900425ba5db982bca51ee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:10.095657 24762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c423c28e5ef430ea2f4624f96ed1219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:10.095973 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c423c28e5ef430ea2f4624f96ed1219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:10.095973 25150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c423c28e5ef430ea2f4624f96ed1219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:10.101883 25271 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Bootstrap starting.
I20260807 02:27:10.103307 25274 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Bootstrap starting.
I20260807 02:27:10.103768 25275 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Bootstrap starting.
I20260807 02:27:10.104380 25271 tablet_bootstrap.cc:654] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:10.105574 25274 tablet_bootstrap.cc:654] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:10.105582 25271 log.cc:824] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:10.105666 25275 tablet_bootstrap.cc:654] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:10.106356 25274 log.cc:824] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:10.106480 25275 log.cc:824] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:10.107774 25271 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: No bootstrap required, opened a new log
I20260807 02:27:10.107986 25271 ts_tablet_manager.cc:1397] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:27:10.108311 25275 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: No bootstrap required, opened a new log
I20260807 02:27:10.108312 25274 tablet_bootstrap.cc:492] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: No bootstrap required, opened a new log
I20260807 02:27:10.108496 25275 ts_tablet_manager.cc:1397] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 02:27:10.108496 25274 ts_tablet_manager.cc:1397] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:27:10.111783 25271 raft_consensus.cc:348] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.112020 25271 raft_consensus.cc:374] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:10.112105 25271 raft_consensus.cc:729] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffa1ce6f2aab4ee49ef367a24dc7fe5a, State: Initialized, Role: FOLLOWER
I20260807 02:27:10.112581 25271 consensus_queue.cc:260] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.112952 25275 raft_consensus.cc:348] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.113205 25275 raft_consensus.cc:374] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:10.113274 25275 raft_consensus.cc:729] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22e4c20e5ce464c802fdac33db47ded, State: Initialized, Role: FOLLOWER
I20260807 02:27:10.113296 24865 heartbeater.cc:503] Master 127.17.53.62:34213 was elected leader, sending a full tablet report...
I20260807 02:27:10.113660 25271 ts_tablet_manager.cc:1428] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:27:10.112952 25274 raft_consensus.cc:348] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.113692 25275 consensus_queue.cc:260] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.113909 25274 raft_consensus.cc:374] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:10.113972 25274 raft_consensus.cc:729] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1314bb661eb446187deac8e4ef64efc, State: Initialized, Role: FOLLOWER
I20260807 02:27:10.114395 25274 consensus_queue.cc:260] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.114703 25275 ts_tablet_manager.cc:1428] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 02:27:10.114718 25058 heartbeater.cc:503] Master 127.17.53.62:34213 was elected leader, sending a full tablet report...
I20260807 02:27:10.115319 25274 ts_tablet_manager.cc:1428] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 02:27:10.115664 25245 heartbeater.cc:503] Master 127.17.53.62:34213 was elected leader, sending a full tablet report...
W20260807 02:27:10.116869 25059 tablet.cc:2369] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:10.181389 24866 tablet.cc:2369] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:10.181945 25283 raft_consensus.cc:482] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:10.182178 25283 raft_consensus.cc:504] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.183413 25283 leader_election.cc:288] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffa1ce6f2aab4ee49ef367a24dc7fe5a (127.17.53.1:46639), c22e4c20e5ce464c802fdac33db47ded (127.17.53.2:38991)
I20260807 02:27:10.185810 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.183652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:46639 (local address 127.17.53.3:40383)
0807 02:27:10.183848 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.183864 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.183972 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.184359 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.184361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.184374 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.184957 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.184960 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.184975 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.185012 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.185461 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.185465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.185469 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.185592 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.185639 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 02:27:10.185834 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.183754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:40383 (local address 127.17.53.1:46639)
0807 02:27:10.183870 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.183877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.184063 (+ 186us) server_negotiation.cc:407] Connection header received
0807 02:27:10.184121 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.184125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.184182 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.184260 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.185263 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.185267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.185270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.185285 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.185311 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.185326 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.185438 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.185640 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.185696 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:27:10.185974 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.184186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:58727 (local address 127.17.53.2:38991)
0807 02:27:10.184337 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.184342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.184357 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:10.184405 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.184411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.184452 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.184563 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.185474 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.185478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.185482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.185499 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.185527 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.185543 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.185645 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.185765 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.185826 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:27:10.185994 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.183695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:38991 (local address 127.17.53.3:58727)
0807 02:27:10.184081 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.184095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.184176 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.184588 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.184591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.184608 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:10.185221 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:10.185224 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:10.185237 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:10.185269 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:10.185646 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:10.185650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:10.185654 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 02:27:10.185745 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 02:27:10.185805 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":148,"threads_started":1}
I20260807 02:27:10.186630 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c423c28e5ef430ea2f4624f96ed1219" candidate_uuid: "c1314bb661eb446187deac8e4ef64efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22e4c20e5ce464c802fdac33db47ded" is_pre_election: true
I20260807 02:27:10.186825 24790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c423c28e5ef430ea2f4624f96ed1219" candidate_uuid: "c1314bb661eb446187deac8e4ef64efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" is_pre_election: true
I20260807 02:27:10.186940 24987 raft_consensus.cc:2454] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1314bb661eb446187deac8e4ef64efc in term 0.
I20260807 02:27:10.187146 24790 raft_consensus.cc:2454] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1314bb661eb446187deac8e4ef64efc in term 0.
I20260807 02:27:10.187412 25079 leader_election.cc:302] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1314bb661eb446187deac8e4ef64efc, c22e4c20e5ce464c802fdac33db47ded; no voters:
I20260807 02:27:10.187695 25283 raft_consensus.cc:2781] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:10.187788 25283 raft_consensus.cc:482] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:10.187845 25283 raft_consensus.cc:3037] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:10.189059 25283 raft_consensus.cc:504] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.189474 25283 leader_election.cc:288] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [CANDIDATE]: Term 1 election: Requested vote from peers ffa1ce6f2aab4ee49ef367a24dc7fe5a (127.17.53.1:46639), c22e4c20e5ce464c802fdac33db47ded (127.17.53.2:38991)
I20260807 02:27:10.189826 24790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c423c28e5ef430ea2f4624f96ed1219" candidate_uuid: "c1314bb661eb446187deac8e4ef64efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a"
I20260807 02:27:10.189872 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c423c28e5ef430ea2f4624f96ed1219" candidate_uuid: "c1314bb661eb446187deac8e4ef64efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c22e4c20e5ce464c802fdac33db47ded"
I20260807 02:27:10.189982 24790 raft_consensus.cc:3037] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:10.190007 24987 raft_consensus.cc:3037] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:10.191095 24987 raft_consensus.cc:2454] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1314bb661eb446187deac8e4ef64efc in term 1.
I20260807 02:27:10.191121 24790 raft_consensus.cc:2454] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1314bb661eb446187deac8e4ef64efc in term 1.
I20260807 02:27:10.191427 25079 leader_election.cc:302] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1314bb661eb446187deac8e4ef64efc, c22e4c20e5ce464c802fdac33db47ded; no voters:
I20260807 02:27:10.191609 25283 raft_consensus.cc:2781] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:10.191910 25283 raft_consensus.cc:686] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 1 LEADER]: Becoming Leader. State: Replica: c1314bb661eb446187deac8e4ef64efc, State: Running, Role: LEADER
I20260807 02:27:10.192297 25283 consensus_queue.cc:237] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } }
I20260807 02:27:10.196130 24574 catalog_manager.cc:5495] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc reported cstate change: term changed from 0 to 1, leader changed from <none> to c1314bb661eb446187deac8e4ef64efc (127.17.53.3). New cstate: current_term: 1 leader_uuid: "c1314bb661eb446187deac8e4ef64efc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1314bb661eb446187deac8e4ef64efc" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 34663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:10.214811 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.213359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.17.53.3:34663)
0807 02:27:10.213495 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.213500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.213523 (+ 23us) server_negotiation.cc:407] Connection header received
0807 02:27:10.213635 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.213638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.213698 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.213771 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.214441 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.214444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.214446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.214457 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.214476 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.214483 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.214556 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.214613 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.214647 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:10.221150 24987 raft_consensus.cc:1261] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 1 FOLLOWER]: Refusing update from remote peer c1314bb661eb446187deac8e4ef64efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:10.221185 24790 raft_consensus.cc:1261] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 1 FOLLOWER]: Refusing update from remote peer c1314bb661eb446187deac8e4ef64efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:10.221904 25285 consensus_queue.cc:1035] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c22e4c20e5ce464c802fdac33db47ded" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:10.222083 25283 consensus_queue.cc:1035] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffa1ce6f2aab4ee49ef367a24dc7fe5a" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 46639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:10.231397 25292 mvcc.cc:205] Tried to move back new op lower bound from 7315741205377683456 to 7315741205270925312. Current Snapshot: MvccSnapshot[applied={T|T < 7315741205377683456}]
I20260807 02:27:10.231601 25291 mvcc.cc:205] Tried to move back new op lower bound from 7315741205377683456 to 7315741205270925312. Current Snapshot: MvccSnapshot[applied={T|T < 7315741205377683456}]
I20260807 02:27:10.238235 24574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:10.238397 24574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:10.240875 24574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 020292a155a14a4fb971fa5ee5c47a3f: Sending DeleteTablet for 3 replicas of tablet 5c423c28e5ef430ea2f4624f96ed1219
I20260807 02:27:10.241622 25150 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:10 UTC) from {username='slave'} at 127.0.0.1:51442
I20260807 02:27:10.241647 24762 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:10 UTC) from {username='slave'} at 127.0.0.1:50272
I20260807 02:27:10.242136 25307 tablet_replica.cc:330] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a: stopping tablet replica
I20260807 02:27:10.242525 25307 raft_consensus.cc:2229] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:10.242589 24959 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c423c28e5ef430ea2f4624f96ed1219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:10 UTC) from {username='slave'} at 127.0.0.1:50394
I20260807 02:27:10.243074 25309 tablet_replica.cc:330] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: stopping tablet replica
I20260807 02:27:10.243081 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 24666
I20260807 02:27:10.243221 25307 raft_consensus.cc:2258] T 5c423c28e5ef430ea2f4624f96ed1219 P ffa1ce6f2aab4ee49ef367a24dc7fe5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:10.243425 25309 raft_consensus.cc:2229] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:10.243772 25309 raft_consensus.cc:2258] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:10.243829 25302 tablet_replica.cc:330] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: stopping tablet replica
I20260807 02:27:10.246114 25302 raft_consensus.cc:2229] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:10.246526 25302 raft_consensus.cc:2258] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:10.247248 25309 ts_tablet_manager.cc:1905] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:10.248039 25302 ts_tablet_manager.cc:1905] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:10.250319 25309 ts_tablet_manager.cc:1918] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:10.250499 25309 log.cc:1196] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/5c423c28e5ef430ea2f4624f96ed1219
I20260807 02:27:10.250983 25309 ts_tablet_manager.cc:1939] T 5c423c28e5ef430ea2f4624f96ed1219 P c22e4c20e5ce464c802fdac33db47ded: Deleting consensus metadata
I20260807 02:27:10.251065 25302 ts_tablet_manager.cc:1918] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:10.251215 25302 log.cc:1196] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/5c423c28e5ef430ea2f4624f96ed1219
I20260807 02:27:10.251665 25302 ts_tablet_manager.cc:1939] T 5c423c28e5ef430ea2f4624f96ed1219 P c1314bb661eb446187deac8e4ef64efc: Deleting consensus metadata
I20260807 02:27:10.252475 24553 catalog_manager.cc:4841] TS c22e4c20e5ce464c802fdac33db47ded (127.17.53.2:38991): tablet 5c423c28e5ef430ea2f4624f96ed1219 (table test-table [id=025376c7c900425ba5db982bca51ee5d]) successfully deleted
I20260807 02:27:10.252974 24554 catalog_manager.cc:4841] TS c1314bb661eb446187deac8e4ef64efc (127.17.53.3:34663): tablet 5c423c28e5ef430ea2f4624f96ed1219 (table test-table [id=025376c7c900425ba5db982bca51ee5d]) successfully deleted
W20260807 02:27:10.261873 24555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:46639 (error 108)
I20260807 02:27:10.262032 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 24873
I20260807 02:27:10.265060 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.264627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:46639 (local address 127.0.0.1:50282)
0807 02:27:10.264864 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.264948 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 02:27:10.265363 24555 catalog_manager.cc:4584] TS ffa1ce6f2aab4ee49ef367a24dc7fe5a (127.17.53.1:46639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c423c28e5ef430ea2f4624f96ed1219: Network error: Client connection negotiation failed: client connection to 127.17.53.1:46639: connect: Connection refused (error 111)
I20260807 02:27:10.275991 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25061
I20260807 02:27:10.289633 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 24527
2026-08-07T02:27:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1073 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 02:27:10.318305 17620 mini_oidc.cc:152] Starting JWKS server
I20260807 02:27:10.319999 17620 webserver.cc:460] Webserver started at https://127.0.0.1:36485/ using document root <none> and password file <none>
I20260807 02:27:10.320195 17620 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 02:27:10.320490 17620 webserver.cc:460] Webserver started at http://127.0.0.1:37259/ using document root <none> and password file <none>
2026-08-07T02:27:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:10Z Disabled control of system clock
I20260807 02:27:10.337153 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:42731
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:40275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:42731
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36485/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/testchainca.cert with env {}
W20260807 02:27:10.453707 25320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:10.453970 25320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:10.454010 25320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:10.457887 25320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:10.457943 25320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:10.457960 25320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:10.457978 25320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:10.457993 25320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 02:27:10.458007 25320 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 02:27:10.458024 25320 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36485/jwks.json
I20260807 02:27:10.463436 25320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:42731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:42731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36485/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:10.464461 25320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:10.465487 25320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:10.473920 25329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:10.473937 25325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:10.473937 25332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:10.473989 25320 server_base.cc:1034] running on GCE node
I20260807 02:27:10.474687 25320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:10.475785 25320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:10.476949 25320 hybrid_clock.cc:648] HybridClock initialized: now 1786069630476921 us; error 38 us; skew 500 ppm
I20260807 02:27:10.478822 25320 webserver.cc:460] Webserver started at http://127.17.53.62:36667/ using document root <none> and password file <none>
I20260807 02:27:10.479460 25320 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:10.479524 25320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:10.479743 25320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:10.481796 25320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "783833c1ce2e3f5f1df839edaab2d619"
server_key_iv: "7097242013cc867144980fb1adff1384"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.482363 25320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "783833c1ce2e3f5f1df839edaab2d619"
server_key_iv: "7097242013cc867144980fb1adff1384"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.486069 25320 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:10.488524 25342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:10.489730 25320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:10.489892 25320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "783833c1ce2e3f5f1df839edaab2d619"
server_key_iv: "7097242013cc867144980fb1adff1384"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.490010 25320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:10.519670 25320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:10.520558 25320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:10.520884 25320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:10.528584 25320 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:42731
I20260807 02:27:10.528596 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:42731 every 8 connection(s)
I20260807 02:27:10.529637 25320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:10.532966 25414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:10.533713 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25320
I20260807 02:27:10.533856 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:10.534139 17620 external_mini_cluster.cc:1428] Setting key 521219ebe404157537d213c78098fc33
I20260807 02:27:10.539884 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3: Bootstrap starting.
I20260807 02:27:10.542594 25414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:10.543386 25414 log.cc:824] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:10.544620 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.535579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.17.53.62:42731)
0807 02:27:10.536259 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.536280 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.536326 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:27:10.537198 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.537223 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.537717 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.538115 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.538646 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.540071 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.541038 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.541899 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.542072 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.542902 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.542907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.542920 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.543319 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.543376 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.543397 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.543508 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.543629 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.543904 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":98,"threads_started":1}
I20260807 02:27:10.545676 25414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3: No bootstrap required, opened a new log
I20260807 02:27:10.547730 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:27:10.549841 25414 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } }
I20260807 02:27:10.550050 25414 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:10.550141 25414 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bdc823dbdd49dcafeabd9f4bec1ec3, State: Initialized, Role: FOLLOWER
I20260807 02:27:10.550645 25414 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } }
I20260807 02:27:10.550825 25414 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:10.550897 25414 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:10.550995 25414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:10.551913 25414 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } }
I20260807 02:27:10.552270 25414 leader_election.cc:302] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6bdc823dbdd49dcafeabd9f4bec1ec3; no voters:
I20260807 02:27:10.552599 25414 leader_election.cc:288] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:10.552824 25425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:10.553169 25425 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [term 1 LEADER]: Becoming Leader. State: Replica: d6bdc823dbdd49dcafeabd9f4bec1ec3, State: Running, Role: LEADER
I20260807 02:27:10.553550 25425 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } }
I20260807 02:27:10.553879 25414 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:10.555279 25427 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6bdc823dbdd49dcafeabd9f4bec1ec3. Latest consensus state: current_term: 1 leader_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } } }
I20260807 02:27:10.555415 25427 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:10.555908 25440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:10.556911 25426 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bdc823dbdd49dcafeabd9f4bec1ec3" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42731 } } }
I20260807 02:27:10.557013 25426 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:10.558641 25440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:10.564124 25440 catalog_manager.cc:1347] Generated new cluster ID: 02bd29d84fa54cdb99edccc83770420e
I20260807 02:27:10.564193 25440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:10.585716 25440 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:10.586802 25440 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:10.597405 25440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6bdc823dbdd49dcafeabd9f4bec1ec3: Generated new TSK 0
I20260807 02:27:10.598189 25440 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 02:27:10.636058 25468 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 02:27:10.636196 25468 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 02:27:10.636910 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.619084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.17.53.62:42731)
0807 02:27:10.619230 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.619235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.619401 (+ 166us) server_negotiation.cc:407] Connection header received
0807 02:27:10.619599 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.619602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.619669 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.619808 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0807 02:27:10.620042 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.621045 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.621951 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.622359 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.622484 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.622801 (+ 317us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 02:27:10.636400 (+ 13599us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 02:27:10.636716 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60786: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 02:27:10.637002 25421 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60786: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 02:27:10.638120 25467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:10.619002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:42731 (local address 127.0.0.1:60786)
0807 02:27:10.619315 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.619328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.619437 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.619816 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.619819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.619826 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 02:27:10.619934 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:10.619939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.621156 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.621159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:10.621822 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:10.621830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.622474 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.622477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:10.622542 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.622585 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 02:27:10.636626 (+ 14041us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 02:27:10.636918 (+ 292us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.53.62:42731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.53.62:42731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36485/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 02:27:10.639030 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25320
2026-08-07T02:27:10Z chronyd exiting
I20260807 02:27:10.684553 17620 test_util.cc:175] -----------------------------------------------
I20260807 02:27:10.684674 17620 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786069602236550-17620-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (367 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 02:27:10.685761 17620 mini_oidc.cc:152] Starting JWKS server
I20260807 02:27:10.686650 17620 webserver.cc:460] Webserver started at https://127.0.0.1:41673/ using document root <none> and password file <none>
I20260807 02:27:10.686770 17620 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 02:27:10.687227 17620 webserver.cc:460] Webserver started at http://127.0.0.1:44365/ using document root <none> and password file <none>
2026-08-07T02:27:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:10Z Disabled control of system clock
I20260807 02:27:10.697901 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40303
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:40303
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41673/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/testchainca.cert with env {}
W20260807 02:27:10.823679 25476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:10.824018 25476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:10.824115 25476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:10.828751 25476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:10.828866 25476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:10.828897 25476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:10.828954 25476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:10.829010 25476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 02:27:10.829052 25476 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 02:27:10.829087 25476 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41673/jwks.json
I20260807 02:27:10.834839 25476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:40303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41673/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:10.835943 25476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:10.837001 25476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:10.845606 25489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:10.845579 25487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:10.845796 25476 server_base.cc:1034] running on GCE node
W20260807 02:27:10.845567 25485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:10.846508 25476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:10.847673 25476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:10.848878 25476 hybrid_clock.cc:648] HybridClock initialized: now 1786069630848836 us; error 45 us; skew 500 ppm
I20260807 02:27:10.850848 25476 webserver.cc:460] Webserver started at http://127.17.53.62:38993/ using document root <none> and password file <none>
I20260807 02:27:10.851460 25476 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:10.851550 25476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:10.851801 25476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:10.854074 25476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "c03b1412510b45c292a57f2e5dcbdf68"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "7ab507cd92cf286ed35315a47cb1f282"
server_key_iv: "112565ed31bbd316cd16baf83b0d8025"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.854808 25476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "c03b1412510b45c292a57f2e5dcbdf68"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "7ab507cd92cf286ed35315a47cb1f282"
server_key_iv: "112565ed31bbd316cd16baf83b0d8025"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.859113 25476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:10.861728 25496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:10.863036 25476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:10.863181 25476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "c03b1412510b45c292a57f2e5dcbdf68"
format_stamp: "Formatted at 2026-08-07 02:27:10 on dist-test-slave-g170"
server_key: "7ab507cd92cf286ed35315a47cb1f282"
server_key_iv: "112565ed31bbd316cd16baf83b0d8025"
server_key_version: "encryptionkey@0"
I20260807 02:27:10.863297 25476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:10.877553 25476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:10.878243 25476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:10.878463 25476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:10.886359 25574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:40303 every 8 connection(s)
I20260807 02:27:10.886358 25476 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:40303
I20260807 02:27:10.887486 25476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:10.891251 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:10.894300 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25476
I20260807 02:27:10.894464 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:10.894774 17620 external_mini_cluster.cc:1428] Setting key 509f2de7b8e50244f9793f8e569bd8a8
I20260807 02:27:10.898517 25575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68: Bootstrap starting.
I20260807 02:27:10.901069 25575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:10.901887 25575 log.cc:824] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:10.904587 25575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68: No bootstrap required, opened a new log
I20260807 02:27:10.905158 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.896403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.17.53.62:40303)
0807 02:27:10.897209 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.897226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.897262 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:27:10.897923 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.897941 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.898356 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.898737 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:10.899179 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.900170 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.901099 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.901621 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.901802 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.902836 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:10.902842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:10.902879 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:10.903437 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:10.903511 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:10.903540 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:10.903649 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:10.903825 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 02:27:10.904110 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":180,"threads_started":1}
I20260807 02:27:10.908543 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:27:10.909493 25575 raft_consensus.cc:348] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } }
I20260807 02:27:10.909803 25575 raft_consensus.cc:374] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:10.909890 25575 raft_consensus.cc:729] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03b1412510b45c292a57f2e5dcbdf68, State: Initialized, Role: FOLLOWER
I20260807 02:27:10.910416 25575 consensus_queue.cc:260] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } }
I20260807 02:27:10.910616 25575 raft_consensus.cc:388] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:10.910705 25575 raft_consensus.cc:482] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:10.910817 25575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:10.911819 25575 raft_consensus.cc:504] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } }
I20260807 02:27:10.912263 25575 leader_election.cc:302] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c03b1412510b45c292a57f2e5dcbdf68; no voters:
I20260807 02:27:10.912640 25575 leader_election.cc:288] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:10.912753 25585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:10.913057 25585 raft_consensus.cc:686] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [term 1 LEADER]: Becoming Leader. State: Replica: c03b1412510b45c292a57f2e5dcbdf68, State: Running, Role: LEADER
I20260807 02:27:10.913419 25585 consensus_queue.cc:237] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } }
I20260807 02:27:10.913993 25575 sys_catalog.cc:564] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:10.915171 25588 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c03b1412510b45c292a57f2e5dcbdf68. Latest consensus state: current_term: 1 leader_uuid: "c03b1412510b45c292a57f2e5dcbdf68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } } }
I20260807 02:27:10.915323 25588 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:10.915723 25602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:10.915679 25587 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c03b1412510b45c292a57f2e5dcbdf68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03b1412510b45c292a57f2e5dcbdf68" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40303 } } }
I20260807 02:27:10.916356 25587 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:10.918601 25602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:10.924477 25602 catalog_manager.cc:1347] Generated new cluster ID: 6a73f7ad2e744c268141b73af179db99
I20260807 02:27:10.924556 25602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:10.937793 25602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:10.938728 25602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:10.956610 25602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c03b1412510b45c292a57f2e5dcbdf68: Generated new TSK 0
I20260807 02:27:10.957350 25602 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 02:27:10.984342 25627 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 02:27:10.984444 25627 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 02:27:10.985265 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:10.969285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.17.53.62:40303)
0807 02:27:10.969450 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:10.969456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:10.969636 (+ 180us) server_negotiation.cc:407] Connection header received
0807 02:27:10.969840 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:10.969842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:10.969893 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:10.970006 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0807 02:27:10.970262 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.970816 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.971745 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.971981 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.972094 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.972486 (+ 392us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 02:27:10.984752 (+ 12266us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 02:27:10.985091 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58076: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 02:27:10.985345 25626 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:10.969224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40303 (local address 127.0.0.1:58076)
0807 02:27:10.969547 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:10.969560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:10.969688 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:10.970021 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:10.970024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:10.970032 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 02:27:10.970154 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:10.970160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.970944 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.970948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:10.971605 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:10.971613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:10.972112 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:10.972116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:10.972210 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:10.972242 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 02:27:10.984990 (+ 12748us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 02:27:10.985158 (+ 168us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.53.62:40303: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":115,"threads_started":1}
W20260807 02:27:10.985351 25580 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58076: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41673/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 02:27:10.986210 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25476
2026-08-07T02:27:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (344 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 02:27:11.030109 17620 mini_oidc.cc:152] Starting JWKS server
I20260807 02:27:11.030983 17620 webserver.cc:460] Webserver started at https://127.0.0.1:42637/ using document root <none> and password file <none>
I20260807 02:27:11.031073 17620 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 02:27:11.031288 17620 webserver.cc:460] Webserver started at http://127.0.0.1:44627/ using document root <none> and password file <none>
2026-08-07T02:27:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:11Z Disabled control of system clock
I20260807 02:27:11.042848 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36127
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36127
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42637/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 02:27:11.157621 25636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:11.157941 25636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:11.158008 25636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:11.162048 25636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:11.162139 25636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:11.162168 25636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:11.162204 25636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:11.162240 25636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 02:27:11.162278 25636 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 02:27:11.162309 25636 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42637/jwks.json
I20260807 02:27:11.167788 25636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42637/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:11.168977 25636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:11.170408 25636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:11.179282 25644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:11.179389 25641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:11.179389 25642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:11.179457 25636 server_base.cc:1034] running on GCE node
I20260807 02:27:11.180209 25636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:11.181740 25636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:11.182925 25636 hybrid_clock.cc:648] HybridClock initialized: now 1786069631182906 us; error 30 us; skew 500 ppm
I20260807 02:27:11.185029 25636 webserver.cc:460] Webserver started at http://127.17.53.62:42373/ using document root <none> and password file <none>
I20260807 02:27:11.185631 25636 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:11.185696 25636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:11.185923 25636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:11.188077 25636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "215f478a14d241b1a2d9634c669ee0e6"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "dfad763244c66d5955efe7f2bd4d61dc"
server_key_iv: "099c3597ccffad39e411665b3b549da2"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.188660 25636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "215f478a14d241b1a2d9634c669ee0e6"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "dfad763244c66d5955efe7f2bd4d61dc"
server_key_iv: "099c3597ccffad39e411665b3b549da2"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.192377 25636 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:11.195091 25655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:11.196353 25636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:11.196493 25636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "215f478a14d241b1a2d9634c669ee0e6"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "dfad763244c66d5955efe7f2bd4d61dc"
server_key_iv: "099c3597ccffad39e411665b3b549da2"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.196607 25636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:11.221069 25636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:11.221824 25636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:11.222030 25636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:11.229521 25636 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36127
I20260807 02:27:11.229544 25727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36127 every 8 connection(s)
I20260807 02:27:11.230494 25636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:11.234174 25728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:11.239291 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25636
I20260807 02:27:11.239395 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:11.239672 17620 external_mini_cluster.cc:1428] Setting key f5875c186eec47737fc5cdd897674bf6
I20260807 02:27:11.240456 25728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6: Bootstrap starting.
I20260807 02:27:11.243436 25728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:11.244482 25728 log.cc:824] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:11.246806 25728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6: No bootstrap required, opened a new log
I20260807 02:27:11.251001 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:11.241172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.17.53.62:36127)
0807 02:27:11.241915 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:11.241936 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:11.241984 (+ 48us) server_negotiation.cc:407] Connection header received
0807 02:27:11.242899 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:11.242924 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:11.243501 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:11.243924 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:11.244445 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.245936 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.247113 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.247778 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.248240 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.249219 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:11.249225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:11.249238 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:11.249645 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:11.249699 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:11.249719 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:11.249829 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:11.249997 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 02:27:11.250262 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":103,"threads_started":1}
I20260807 02:27:11.251230 25728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } }
I20260807 02:27:11.251499 25728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:11.251590 25728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215f478a14d241b1a2d9634c669ee0e6, State: Initialized, Role: FOLLOWER
I20260807 02:27:11.252043 25728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } }
I20260807 02:27:11.252203 25728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:11.252291 25728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:11.252394 25728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:11.253549 25728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } }
I20260807 02:27:11.253916 25728 leader_election.cc:302] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 215f478a14d241b1a2d9634c669ee0e6; no voters:
I20260807 02:27:11.254045 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:27:11.254297 25728 leader_election.cc:288] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:11.254375 25735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:11.254570 25735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [term 1 LEADER]: Becoming Leader. State: Replica: 215f478a14d241b1a2d9634c669ee0e6, State: Running, Role: LEADER
I20260807 02:27:11.255102 25735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } }
I20260807 02:27:11.255671 25728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:11.256601 25735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215f478a14d241b1a2d9634c669ee0e6. Latest consensus state: current_term: 1 leader_uuid: "215f478a14d241b1a2d9634c669ee0e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } } }
I20260807 02:27:11.256755 25735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:11.257059 25736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "215f478a14d241b1a2d9634c669ee0e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215f478a14d241b1a2d9634c669ee0e6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36127 } } }
I20260807 02:27:11.257215 25736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:11.257279 25742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:11.260983 25742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:11.266861 25742 catalog_manager.cc:1347] Generated new cluster ID: 62e9010ed8944b65a023e0009bf01b16
I20260807 02:27:11.266947 25742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:11.283458 25742 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:11.284457 25742 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:11.299209 25742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 215f478a14d241b1a2d9634c669ee0e6: Generated new TSK 0
I20260807 02:27:11.299942 25742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:11.322280 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:11.316290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.17.53.62:36127)
0807 02:27:11.316433 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:11.316438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:11.316587 (+ 149us) server_negotiation.cc:407] Connection header received
0807 02:27:11.316725 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:11.316728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:11.316807 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:11.316928 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0807 02:27:11.317135 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.317967 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.318840 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.319190 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.319297 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.319549 (+ 252us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 02:27:11.321899 (+ 2350us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42637/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 02:27:11.322140 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46158: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42637/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 02:27:11.322364 25733 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46158: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42637/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 02:27:11.323866 25769 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:11.316170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36127 (local address 127.0.0.1:46158)
0807 02:27:11.316499 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:11.316512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:11.316607 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:11.316925 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:11.316928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:11.316935 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 02:27:11.317034 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:11.317039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.318095 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.318098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:11.318724 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:11.318732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.319302 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.319305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:11.319366 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.319388 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 02:27:11.322062 (+ 2674us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42637/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 02:27:11.323690 (+ 1628us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.53.62:36127: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42637/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1445,"thread_start_us":115,"threads_started":1}
I20260807 02:27:11.324312 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25636
E20260807 02:27:11.335280 25770 webserver.cc:533] Webserver: sslize failed:
E20260807 02:27:11.335369 25770 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T02:27:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T02:27:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:11Z Disabled control of system clock
I20260807 02:27:11.380268 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35899
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35899
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/testchainca.cert with env {}
W20260807 02:27:11.496881 25774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:11.497116 25774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:11.497153 25774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:11.501000 25774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:11.501042 25774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:11.501057 25774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:11.501073 25774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:11.501088 25774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 02:27:11.501107 25774 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 02:27:11.501128 25774 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 02:27:11.506299 25774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:11.507246 25774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:11.508183 25774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:11.516482 25783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:11.516462 25779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:11.516451 25780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:11.516717 25774 server_base.cc:1034] running on GCE node
I20260807 02:27:11.517395 25774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:11.518409 25774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:11.519599 25774 hybrid_clock.cc:648] HybridClock initialized: now 1786069631519578 us; error 32 us; skew 500 ppm
I20260807 02:27:11.525709 25774 webserver.cc:460] Webserver started at http://127.17.53.62:42903/ using document root <none> and password file <none>
I20260807 02:27:11.526309 25774 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:11.526402 25774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:11.526629 25774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:11.528657 25774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "6008c5ab4e9e4af2997995d2e7365531"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "85f60705fbb8cf8e23fe7a25b90c108c"
server_key_iv: "91d1e8d200998c5fda4e4874aa3f66ac"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.529295 25774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "6008c5ab4e9e4af2997995d2e7365531"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "85f60705fbb8cf8e23fe7a25b90c108c"
server_key_iv: "91d1e8d200998c5fda4e4874aa3f66ac"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.532948 25774 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:11.536625 25798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:11.537910 25774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 02:27:11.538054 25774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "6008c5ab4e9e4af2997995d2e7365531"
format_stamp: "Formatted at 2026-08-07 02:27:11 on dist-test-slave-g170"
server_key: "85f60705fbb8cf8e23fe7a25b90c108c"
server_key_iv: "91d1e8d200998c5fda4e4874aa3f66ac"
server_key_version: "encryptionkey@0"
I20260807 02:27:11.538163 25774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:11.548558 25774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:11.549227 25774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:11.549413 25774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:11.556350 25774 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35899
I20260807 02:27:11.556372 25883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35899 every 8 connection(s)
I20260807 02:27:11.557307 25774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:11.560213 25884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:11.566520 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25774
I20260807 02:27:11.566668 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:11.566707 25884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531: Bootstrap starting.
I20260807 02:27:11.567008 17620 external_mini_cluster.cc:1428] Setting key afdc2d2fd192e5a409d4500f93263aa6
I20260807 02:27:11.569626 25884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:11.570567 25884 log.cc:824] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:11.572860 25884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531: No bootstrap required, opened a new log
I20260807 02:27:11.576495 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:11.568318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.17.53.62:35899)
0807 02:27:11.568972 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:11.568986 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:11.569020 (+ 34us) server_negotiation.cc:407] Connection header received
0807 02:27:11.569612 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:11.569627 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:11.570078 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:11.570345 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:11.570771 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.571785 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.573107 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.573557 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.573674 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.574602 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:11.574607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:11.574620 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:11.575021 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:11.575086 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:11.575106 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:11.575182 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:11.575606 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 02:27:11.575816 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":270,"thread_start_us":128,"threads_started":1}
I20260807 02:27:11.577294 25884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } }
I20260807 02:27:11.577577 25884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:11.577642 25884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6008c5ab4e9e4af2997995d2e7365531, State: Initialized, Role: FOLLOWER
I20260807 02:27:11.578114 25884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } }
I20260807 02:27:11.578306 25884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:11.578424 25884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:11.578528 25884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:11.579536 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:27:11.579682 25884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } }
I20260807 02:27:11.580106 25884 leader_election.cc:302] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6008c5ab4e9e4af2997995d2e7365531; no voters:
I20260807 02:27:11.580493 25884 leader_election.cc:288] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:11.580746 25893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:11.581051 25893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [term 1 LEADER]: Becoming Leader. State: Replica: 6008c5ab4e9e4af2997995d2e7365531, State: Running, Role: LEADER
I20260807 02:27:11.581410 25893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } }
I20260807 02:27:11.582312 25884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:11.582901 25894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6008c5ab4e9e4af2997995d2e7365531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } } }
I20260807 02:27:11.583029 25894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:11.583372 25895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6008c5ab4e9e4af2997995d2e7365531. Latest consensus state: current_term: 1 leader_uuid: "6008c5ab4e9e4af2997995d2e7365531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6008c5ab4e9e4af2997995d2e7365531" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35899 } } }
I20260807 02:27:11.583465 25895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:11.583570 25900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:11.586599 25900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:11.591727 25900 catalog_manager.cc:1347] Generated new cluster ID: 490b2d7f1d654cecb5d6c0fc835ee5b7
I20260807 02:27:11.591789 25900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:11.602427 25900 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:11.603636 25900 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:11.609380 25900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6008c5ab4e9e4af2997995d2e7365531: Generated new TSK 0
I20260807 02:27:11.610189 25900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:12.126441 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:11.619972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.17.53.62:35899)
0807 02:27:11.620115 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:11.620122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:11.620273 (+ 151us) server_negotiation.cc:407] Connection header received
0807 02:27:11.620430 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:11.620434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:11.620494 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:11.620621 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0807 02:27:11.620869 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.621699 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.622563 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.623002 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.623104 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.623408 (+ 304us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 02:27:12.125843 (+502435us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 02:27:12.126264 (+ 421us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43896: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 02:27:12.126552 25932 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:11.619884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35899 (local address 127.0.0.1:43896)
0807 02:27:11.620182 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:11.620195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:11.620294 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:11.620624 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:11.620626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:11.620633 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 02:27:11.620727 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:11.620732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.621837 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.621839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:11.622452 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:11.622459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:11.623136 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:11.623139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:11.623207 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:11.623231 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 02:27:12.126113 (+502882us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 02:27:12.126352 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.53.62:35899: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
W20260807 02:27:12.126556 25887 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43896: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260807 02:27:12.127408 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25774
2026-08-07T02:27:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (788 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (145 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (136 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:12 dist-test-slave-g170 krb5kdc[25947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:12 dist-test-slave-g170 krb5kdc[25947](info): set up 2 sockets
Aug 07 02:27:12 dist-test-slave-g170 krb5kdc[25947](info): commencing operation
krb5kdc: starting...
W20260807 02:27:13.674146 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:13 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:13.804108 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38769
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:38769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:13.919816 25965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:13.920132 25965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:13.920214 25965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:13.924291 25965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:13.924386 25965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:13.924425 25965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:13.924472 25965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:13.924507 25965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:13.929873 25965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:38769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:13.930975 25965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:13.932054 25965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:13.940879 25971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:13.940872 25977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:13.940924 25970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:13.941249 25965 server_base.cc:1034] running on GCE node
I20260807 02:27:13.941814 25965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:13.942899 25965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:13.944092 25965 hybrid_clock.cc:648] HybridClock initialized: now 1786069633944069 us; error 36 us; skew 500 ppm
Aug 07 02:27:13 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069633, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:13.946981 25965 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:13.947876 25965 webserver.cc:460] Webserver started at http://127.17.53.62:38251/ using document root <none> and password file <none>
I20260807 02:27:13.948436 25965 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:13.948523 25965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:13.948753 25965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:13.950875 25965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe"
format_stamp: "Formatted at 2026-08-07 02:27:13 on dist-test-slave-g170"
server_key: "3d70cc7e6ba80ef479f2ac90769757d8"
server_key_iv: "23afe931a2374f14ce9a36f52c9e86dd"
server_key_version: "encryptionkey@0"
I20260807 02:27:13.951424 25965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe"
format_stamp: "Formatted at 2026-08-07 02:27:13 on dist-test-slave-g170"
server_key: "3d70cc7e6ba80ef479f2ac90769757d8"
server_key_iv: "23afe931a2374f14ce9a36f52c9e86dd"
server_key_version: "encryptionkey@0"
I20260807 02:27:13.955540 25965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 02:27:13.958189 25988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:13.959745 25965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:27:13.959959 25965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe"
format_stamp: "Formatted at 2026-08-07 02:27:13 on dist-test-slave-g170"
server_key: "3d70cc7e6ba80ef479f2ac90769757d8"
server_key_iv: "23afe931a2374f14ce9a36f52c9e86dd"
server_key_version: "encryptionkey@0"
I20260807 02:27:13.960090 25965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:13.972642 25965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:13.976311 25965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:13.976533 25965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:13.984186 25965 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:38769
I20260807 02:27:13.984220 26068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:38769 every 8 connection(s)
I20260807 02:27:13.985235 25965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:13.988134 26072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:13.991098 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 25965
I20260807 02:27:13.991219 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:13.991544 17620 external_mini_cluster.cc:1428] Setting key 175ae654418224de53d886ba5cbd7df2
I20260807 02:27:13.994828 26072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe: Bootstrap starting.
I20260807 02:27:13.997905 26072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:13.998745 26072 log.cc:824] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:14.001013 26072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe: No bootstrap required, opened a new log
Aug 07 02:27:13 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:14.005312 26072 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } }
I20260807 02:27:14.005625 26072 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:14.005683 26072 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb9b3e4d7c74dbfa1d274a0288e53fe, State: Initialized, Role: FOLLOWER
I20260807 02:27:14.006233 26072 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } }
I20260807 02:27:14.006417 26072 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:14.006501 26072 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:14.006609 26072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:14.007905 26072 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } }
I20260807 02:27:14.008283 26072 leader_election.cc:302] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdb9b3e4d7c74dbfa1d274a0288e53fe; no voters:
I20260807 02:27:14.008656 26072 leader_election.cc:288] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:14.008813 26082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:14.009066 26082 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [term 1 LEADER]: Becoming Leader. State: Replica: bdb9b3e4d7c74dbfa1d274a0288e53fe, State: Running, Role: LEADER
I20260807 02:27:14.009335 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:13.993176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.17.53.62:38769)
0807 02:27:13.993682 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:13.993707 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:13.993755 (+ 48us) server_negotiation.cc:407] Connection header received
0807 02:27:13.994603 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:13.994643 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:13.995086 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:13.995474 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:13.996266 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:13.997949 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:13.999020 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:13.999627 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:13.999806 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.002717 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.002724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.002742 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.002776 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.005596 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.006255 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.006262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.006269 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.006429 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.006807 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.006811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.006812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.007351 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.007569 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.007870 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.008198 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":146,"threads_started":1}
I20260807 02:27:14.009430 26082 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } }
I20260807 02:27:14.010002 26072 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:14.011204 26083 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } } }
I20260807 02:27:14.011360 26083 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:14.011529 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb9b3e4d7c74dbfa1d274a0288e53fe. Latest consensus state: current_term: 1 leader_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb9b3e4d7c74dbfa1d274a0288e53fe" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38769 } } }
I20260807 02:27:14.011677 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:14.012380 26094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:14.015538 26094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:14.021390 26094 catalog_manager.cc:1347] Generated new cluster ID: ee1a337f972c4915b8674fde711b7391
I20260807 02:27:14.021461 26094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:14.029197 26094 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:14.030452 26094 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:14.038132 26094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdb9b3e4d7c74dbfa1d274a0288e53fe: Generated new TSK 0
I20260807 02:27:14.038946 26094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:14.075919 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:14.196847 26112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:14.197163 26112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:14.197211 26112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:14.201618 26112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:14.201704 26112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:14.201792 26112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:14.207261 26112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:14.208381 26112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:14.209528 26112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:14.219583 26124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:14.219601 26127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.219735 26112 server_base.cc:1034] running on GCE node
W20260807 02:27:14.219918 26122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.220362 26112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:14.221213 26112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:14.222375 26112 hybrid_clock.cc:648] HybridClock initialized: now 1786069634222343 us; error 47 us; skew 500 ppm
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.225356 26112 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:14.226315 26112 webserver.cc:460] Webserver started at http://127.17.53.1:38659/ using document root <none> and password file <none>
I20260807 02:27:14.226925 26112 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:14.226991 26112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:14.227262 26112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:14.229501 26112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "0956baa0f62a41cfac6a29140e64b423"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "7b27d12b2c6a64c37fc6f65b86fd3a33"
server_key_iv: "bfc8970e494396d2fa3d0810f76cbd18"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.230073 26112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "0956baa0f62a41cfac6a29140e64b423"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "7b27d12b2c6a64c37fc6f65b86fd3a33"
server_key_iv: "bfc8970e494396d2fa3d0810f76cbd18"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.233994 26112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 02:27:14.236583 26136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.237962 26112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:14.238106 26112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "0956baa0f62a41cfac6a29140e64b423"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "7b27d12b2c6a64c37fc6f65b86fd3a33"
server_key_iv: "bfc8970e494396d2fa3d0810f76cbd18"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.238236 26112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:14.262045 26112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:14.266323 26112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:14.266578 26112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:14.267331 26112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:14.268656 26112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:14.268741 26112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.268846 26112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:14.268884 26112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.280483 26112 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:34805
I20260807 02:27:14.280508 26298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:34805 every 8 connection(s)
I20260807 02:27:14.281648 26112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:14.284245 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 26112
I20260807 02:27:14.284407 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:14.284747 17620 external_mini_cluster.cc:1428] Setting key 510dfb0106404ee955ecdc71acd71019
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:27:14.298705 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.283637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:33179 (local address 127.17.53.62:38769)
0807 02:27:14.283827 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.283831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.284899 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 02:27:14.286675 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.286677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.286730 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.286800 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.288561 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.289184 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.290602 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.290878 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.290979 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.294398 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.294401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.294403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.294430 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.296054 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.296721 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.296725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.296726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.296868 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.297232 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.297235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.297236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.297437 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.297524 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.298397 (+ 873us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.298542 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:27:14.299727 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.284052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38769 (local address 127.17.53.1:33179)
0807 02:27:14.284631 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:14.284747 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:14.286409 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:14.287010 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:14.287024 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:14.287599 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:14.288371 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.288397 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.289340 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.289343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.290424 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.290441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.291064 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.291071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.291331 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.292068 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:14.292095 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:14.294169 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:14.296194 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.296202 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.296228 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.296621 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.296969 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.296971 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.296973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.297128 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.297568 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:14.297574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:14.298108 (+ 534us) client_negotiation.cc:769] Sending connection context
0807 02:27:14.298302 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 02:27:14.298609 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":137,"threads_started":1}
I20260807 02:27:14.301455 26300 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38769
I20260807 02:27:14.301802 26300 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:14.302368 26300 heartbeater.cc:511] Master 127.17.53.62:38769 requested a full tablet report, sending...
I20260807 02:27:14.304128 26015 ts_manager.cc:194] Registered new tserver with Master: 0956baa0f62a41cfac6a29140e64b423 (127.17.53.1:34805)
I20260807 02:27:14.305955 26015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:33179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:14.321353 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:14.447173 26305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:14.447427 26305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:14.447469 26305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:14.451517 26305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:14.451571 26305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:14.451664 26305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:14.456729 26305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:14.457858 26305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:14.458875 26305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:14.469062 26315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.469180 26305 server_base.cc:1034] running on GCE node
W20260807 02:27:14.469091 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:14.469045 26319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.469717 26305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:14.470391 26305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:14.471592 26305 hybrid_clock.cc:648] HybridClock initialized: now 1786069634471571 us; error 32 us; skew 500 ppm
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.474520 26305 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:14.475340 26305 webserver.cc:460] Webserver started at http://127.17.53.2:43159/ using document root <none> and password file <none>
I20260807 02:27:14.475864 26305 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:14.475922 26305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:14.476089 26305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:14.478119 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "5534ad8d9a47450cb504aeae34edb30b"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6bbbccb1baad9a6a6251048e2a48c2c3"
server_key_iv: "f702371fdde49c3dace937dbecf3f94f"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.478709 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "5534ad8d9a47450cb504aeae34edb30b"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6bbbccb1baad9a6a6251048e2a48c2c3"
server_key_iv: "f702371fdde49c3dace937dbecf3f94f"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.483016 26305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:14.485879 26329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.487157 26305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:14.487309 26305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "5534ad8d9a47450cb504aeae34edb30b"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6bbbccb1baad9a6a6251048e2a48c2c3"
server_key_iv: "f702371fdde49c3dace937dbecf3f94f"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.487421 26305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:14.506804 26305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:14.510874 26305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:14.511096 26305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:14.511761 26305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:14.512821 26305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:14.512882 26305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.512948 26305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:14.512984 26305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.522229 26305 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:37189
I20260807 02:27:14.522249 26494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:37189 every 8 connection(s)
I20260807 02:27:14.523167 26305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:14.529619 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 26305
I20260807 02:27:14.529747 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:14.530063 17620 external_mini_cluster.cc:1428] Setting key 4191e69b9087b040487b2ea40062e8e9
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:14.538311 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.525063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:51939 (local address 127.17.53.62:38769)
0807 02:27:14.525211 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.525215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.526168 (+ 953us) server_negotiation.cc:407] Connection header received
0807 02:27:14.527709 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.527713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.527771 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.527881 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.529149 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.529897 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.531308 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.531574 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.531680 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.534332 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.534336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.534339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.534363 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.535838 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.536521 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.536524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.536525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.536602 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.537044 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.537046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.537048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.537224 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.537313 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.538036 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.538142 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:27:14.539600 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.525493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38769 (local address 127.17.53.2:51939)
0807 02:27:14.525996 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:14.526052 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:14.527482 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:14.528062 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:14.528070 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:14.528430 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:14.528999 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.529016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.530058 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.530061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.531162 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.531173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.531727 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.531730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.531899 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.532586 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:14.532617 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:14.534155 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:14.535963 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.535972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.535990 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.536395 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.536697 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.536704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.536709 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.536935 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.537356 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:14.537363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:14.537742 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 02:27:14.537972 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 02:27:14.538418 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":124,"threads_started":1}
I20260807 02:27:14.540881 26495 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38769
I20260807 02:27:14.541148 26495 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:14.541729 26495 heartbeater.cc:511] Master 127.17.53.62:38769 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:27:14.542984 26015 ts_manager.cc:194] Registered new tserver with Master: 5534ad8d9a47450cb504aeae34edb30b (127.17.53.2:37189)
I20260807 02:27:14.543560 26015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:51939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:14.561848 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:14.679494 26502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:14.679749 26502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:14.679791 26502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:14.683933 26502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:14.683987 26502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:14.684042 26502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:14.689261 26502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:14.690205 26502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:14.691272 26502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:14.701094 26508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:14.701100 26511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.701182 26502 server_base.cc:1034] running on GCE node
W20260807 02:27:14.701341 26507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:14.701805 26502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:14.702461 26502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:14.703644 26502 hybrid_clock.cc:648] HybridClock initialized: now 1786069634703622 us; error 33 us; skew 500 ppm
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.706897 26502 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:14.707867 26502 webserver.cc:460] Webserver started at http://127.17.53.3:46589/ using document root <none> and password file <none>
I20260807 02:27:14.708521 26502 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:14.708627 26502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:14.708953 26502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:14.711364 26502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "5452bd169dae4bf683d8dcba720ada84"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6ebab23b6e24f7b2402ee624a5a1d269"
server_key_iv: "0c9ed71d990213c5fd949b00ef170531"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.711987 26502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "5452bd169dae4bf683d8dcba720ada84"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6ebab23b6e24f7b2402ee624a5a1d269"
server_key_iv: "0c9ed71d990213c5fd949b00ef170531"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.716135 26502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:27:14.718858 26524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.720119 26502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:14.720269 26502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "5452bd169dae4bf683d8dcba720ada84"
format_stamp: "Formatted at 2026-08-07 02:27:14 on dist-test-slave-g170"
server_key: "6ebab23b6e24f7b2402ee624a5a1d269"
server_key_iv: "0c9ed71d990213c5fd949b00ef170531"
server_key_version: "encryptionkey@0"
I20260807 02:27:14.720388 26502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:14.734335 26502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:14.737974 26502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:14.738199 26502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:14.738886 26502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:14.739856 26502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:14.739909 26502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.739954 26502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:14.740010 26502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:14.750671 26502 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:40069
I20260807 02:27:14.750766 26690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:40069 every 8 connection(s)
I20260807 02:27:14.751729 26502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:14.759588 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 26502
I20260807 02:27:14.759722 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:14.760176 17620 external_mini_cluster.cc:1428] Setting key 44909811440edd986a04cc0e8f8bf843
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:14.768010 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.753989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:55265 (local address 127.17.53.62:38769)
0807 02:27:14.754170 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.754175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.755182 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 02:27:14.756665 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.756668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.756724 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.756891 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.758133 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.758727 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.760040 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.760423 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.760563 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.764106 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.764111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.764115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.764141 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.765896 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.766444 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.766448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.766449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.766532 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.766898 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.766901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.766902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.767084 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.767165 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.767722 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.767871 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 02:27:14.768994 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.754473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38769 (local address 127.17.53.3:55265)
0807 02:27:14.755007 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:14.755054 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:14.756439 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:14.757056 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:14.757063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:14.757426 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:14.757954 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.757968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.758861 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.758863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.759908 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:14.759919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.760625 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.760630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:14.760941 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.761713 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:14.761731 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:14.763809 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:14.766029 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.766034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.766046 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.766330 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.766628 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:14.766630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:14.766632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:14.766809 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:14.767208 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:14.767214 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:14.767482 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 02:27:14.767664 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 02:27:14.768013 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":135,"threads_started":1}
I20260807 02:27:14.770232 26692 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38769
I20260807 02:27:14.770506 26692 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:14.771098 26692 heartbeater.cc:511] Master 127.17.53.62:38769 requested a full tablet report, sending...
I20260807 02:27:14.772387 26015 ts_manager.cc:194] Registered new tserver with Master: 5452bd169dae4bf683d8dcba720ada84 (127.17.53.3:40069)
I20260807 02:27:14.773089 26015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:55265
I20260807 02:27:14.775082 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/test-admin.keytab.
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.793958 17620 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 02:27:14.797076 26703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:14.795438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.17.53.1:34805)
0807 02:27:14.795773 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.795806 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.795842 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:27:14.795912 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.795917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.796108 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.796276 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.796845 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59788: BlockingRecv error: recv got EOF from 127.0.0.1:59788 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
W20260807 02:27:14.797128 17620 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc' of type FILE
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.798226 17620 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.799252 17620 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:27:14.810438 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.800326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.17.53.1:34805)
0807 02:27:14.800464 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.800468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.800485 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:14.800602 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.800604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.800645 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.800715 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.801458 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.802282 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.803167 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.803578 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.803689 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.806182 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.806191 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.806205 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.806228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.808288 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.808873 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.808879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.808880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.808967 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.809279 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.809282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.809284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.809556 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.809741 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.810073 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.810261 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 02:27:14.814620 26703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:14.813691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.17.53.1:34805)
0807 02:27:14.813859 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.813862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.813879 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:14.813991 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.813993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.814030 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.814099 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.814501 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59806: BlockingRecv error: recv got EOF from 127.0.0.1:59806 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 02:27:14.814733 17620 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc' of type FILE
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.815853 17620 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.817041 17620 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:27:14.826921 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.817919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.17.53.1:34805)
0807 02:27:14.818049 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.818053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.818166 (+ 113us) server_negotiation.cc:407] Connection header received
0807 02:27:14.818332 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.818335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.818403 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.818494 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.819044 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.819594 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.820564 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.820944 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.821070 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.823461 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.823465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.823468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.823495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.825247 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.825676 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.825680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.825682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.825775 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.826060 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.826063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.826065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.826245 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.826412 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.826632 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.826742 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 02:27:14.829943 26703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:14.828927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.17.53.1:34805)
0807 02:27:14.829053 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.829057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.829089 (+ 32us) server_negotiation.cc:407] Connection header received
0807 02:27:14.829198 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.829200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.829236 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.829308 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.829839 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59822: BlockingRecv error: recv got EOF from 127.0.0.1:59822 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 02:27:14.829995 17620 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786069602236550-17620-0/krb5kdc/krb5cc' of type FILE
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.831074 17620 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:14.832026 17620 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[25947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:27:14.840569 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:14.832865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.17.53.1:34805)
0807 02:27:14.832986 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:14.832990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:14.833145 (+ 155us) server_negotiation.cc:407] Connection header received
0807 02:27:14.833265 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:14.833267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:14.833310 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:14.833379 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:14.833859 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.834356 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.835154 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:14.835389 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:14.835493 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:14.837734 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:14.837738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:14.837740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:14.837761 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:14.839043 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.839464 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.839466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.839467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.839553 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:14.839826 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:14.839829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:14.839830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:14.839992 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:14.840184 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:14.840273 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 02:27:14.840369 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:27:14.841607 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 26112
I20260807 02:27:14.858826 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 26305
I20260807 02:27:14.871160 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 26502
I20260807 02:27:14.889454 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 25965
2026-08-07T02:27:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2479 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[26726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[26726](info): set up 2 sockets
Aug 07 02:27:14 dist-test-slave-g170 krb5kdc[26726](info): commencing operation
krb5kdc: starting...
W20260807 02:27:16.175493 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab.
Entry for principal oryx/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab.
I20260807 02:27:16.316139 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40831
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab
--principal=oryx/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:16.434103 26746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:16.434392 26746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:16.434434 26746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:16.438347 26746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:16.438412 26746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:16.438431 26746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:16.438448 26746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:16.438463 26746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:16.443740 26746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:40831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab
--principal=oryx/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:16.444787 26746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:16.445792 26746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:16.454476 26767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:16.454527 26755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:16.454480 26763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:16.454702 26746 server_base.cc:1034] running on GCE node
I20260807 02:27:16.455806 26746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:16.457420 26746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:16.458606 26746 hybrid_clock.cc:648] HybridClock initialized: now 1786069636458557 us; error 59 us; skew 500 ppm
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:16.461758 26746 init.cc:377] Logged in from keytab as oryx/127.17.53.62@KRBTEST.COM (short username oryx)
I20260807 02:27:16.462769 26746 webserver.cc:460] Webserver started at http://127.17.53.62:35981/ using document root <none> and password file <none>
I20260807 02:27:16.463384 26746 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:16.463454 26746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:16.463742 26746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:16.466053 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "dd0776d83279461e9688ce1ea8d11ae1"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "568ae6803417ae5fb2b1ef269cd791fe"
server_key_iv: "e83bfe1b3dd0a89bf17ad9b269c93ca2"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.466684 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "dd0776d83279461e9688ce1ea8d11ae1"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "568ae6803417ae5fb2b1ef269cd791fe"
server_key_iv: "e83bfe1b3dd0a89bf17ad9b269c93ca2"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.470870 26746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:16.473524 26775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:16.474867 26746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 02:27:16.474999 26746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "dd0776d83279461e9688ce1ea8d11ae1"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "568ae6803417ae5fb2b1ef269cd791fe"
server_key_iv: "e83bfe1b3dd0a89bf17ad9b269c93ca2"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.475107 26746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:16.486445 26746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:16.489687 26746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:16.489897 26746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:16.497650 26746 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:40831
I20260807 02:27:16.498021 26843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:40831 every 8 connection(s)
I20260807 02:27:16.498852 26746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:16.501930 26844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:16.504527 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 26746
I20260807 02:27:16.504681 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:16.505033 17620 external_mini_cluster.cc:1428] Setting key 7ca0ccaa1e3d8475989bc50cb6fdbbd4
I20260807 02:27:16.508406 26844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Bootstrap starting.
I20260807 02:27:16.510777 26844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:16.511548 26844 log.cc:824] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:16.513775 26844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: No bootstrap required, opened a new log
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:16.518481 26844 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } }
I20260807 02:27:16.518781 26844 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:16.518836 26844 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0776d83279461e9688ce1ea8d11ae1, State: Initialized, Role: FOLLOWER
I20260807 02:27:16.519348 26844 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } }
I20260807 02:27:16.519536 26844 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:16.519611 26844 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:16.519704 26844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:16.520891 26844 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } }
I20260807 02:27:16.520959 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:16.506635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.17.53.62:40831)
0807 02:27:16.507165 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:16.507182 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:16.507226 (+ 44us) server_negotiation.cc:407] Connection header received
0807 02:27:16.507922 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:16.507941 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:16.508294 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:16.508685 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:16.509648 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.510794 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.511767 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.512351 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.512581 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:16.515204 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:16.515213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:16.515239 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:16.515275 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:16.517514 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:16.517997 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:16.518003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:16.518010 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:16.518154 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:16.518486 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:16.518491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:16.518495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:16.519024 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:16.519240 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:16.519564 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 02:27:16.519895 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":153,"threads_started":1}
I20260807 02:27:16.521281 26844 leader_election.cc:302] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd0776d83279461e9688ce1ea8d11ae1; no voters:
I20260807 02:27:16.521642 26844 leader_election.cc:288] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:16.521811 26855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:16.522063 26855 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 LEADER]: Becoming Leader. State: Replica: dd0776d83279461e9688ce1ea8d11ae1, State: Running, Role: LEADER
I20260807 02:27:16.522379 26855 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } }
I20260807 02:27:16.522810 26844 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:16.523620 26855 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } }
I20260807 02:27:16.523756 26855 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:16.523972 26855 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd0776d83279461e9688ce1ea8d11ae1. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } }
I20260807 02:27:16.524066 26855 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:16.524353 26867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:16.527712 26867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:16.533793 26867 catalog_manager.cc:1347] Generated new cluster ID: 2b996ed1245646dd86900b0d1d784300
I20260807 02:27:16.533861 26867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:16.553251 26867 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:16.554328 26867 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:16.559834 26867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Generated new TSK 0
I20260807 02:27:16.560509 26867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
I20260807 02:27:16.596072 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:16.734349 26882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:16.734611 26882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:16.734663 26882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:16.738646 26882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:16.738703 26882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:16.738760 26882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:16.743966 26882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:16.744961 26882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:16.745960 26882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:16.756098 26895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:16.756115 26891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:16.756296 26882 server_base.cc:1034] running on GCE node
W20260807 02:27:16.756134 26890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:16.756892 26882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:16.757563 26882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:16.758852 26882 hybrid_clock.cc:648] HybridClock initialized: now 1786069636758821 us; error 46 us; skew 500 ppm
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:16.764591 26882 init.cc:377] Logged in from keytab as oryx/127.17.53.1@KRBTEST.COM (short username oryx)
I20260807 02:27:16.765635 26882 webserver.cc:460] Webserver started at http://127.17.53.1:35539/ using document root <none> and password file <none>
I20260807 02:27:16.766263 26882 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:16.766360 26882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:16.766618 26882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:16.768893 26882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "70bf2b351775d4d97490614a4d11a0a4"
server_key_iv: "8fd2defdac12fc97050bd1f87353827d"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.769493 26882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "70bf2b351775d4d97490614a4d11a0a4"
server_key_iv: "8fd2defdac12fc97050bd1f87353827d"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.773496 26882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:16.776223 26906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:16.777748 26882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:27:16.777944 26882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "70bf2b351775d4d97490614a4d11a0a4"
server_key_iv: "8fd2defdac12fc97050bd1f87353827d"
server_key_version: "encryptionkey@0"
I20260807 02:27:16.778071 26882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:16.793643 26882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:16.797106 26882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:16.797333 26882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:16.797984 26882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:16.798883 26882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:16.798957 26882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:16.799031 26882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:16.799075 26882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:16.809298 26882 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:36661
I20260807 02:27:16.809329 27081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:36661 every 8 connection(s)
I20260807 02:27:16.810263 26882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:16.815379 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 26882
I20260807 02:27:16.815493 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:16.815827 17620 external_mini_cluster.cc:1428] Setting key 5a95011f3d5ffef35eba4b60673b8a8e
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:16.826838 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:16.812180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:34731 (local address 127.17.53.62:40831)
0807 02:27:16.812339 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:16.812343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:16.813254 (+ 911us) server_negotiation.cc:407] Connection header received
0807 02:27:16.814851 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:16.814854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:16.814919 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:16.815032 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:16.816554 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.817329 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.819219 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.819516 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.819627 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:16.822769 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:16.822772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:16.822774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:16.822800 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:16.824501 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:16.825138 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:16.825142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:16.825144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:16.825232 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:16.825598 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:16.825602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:16.825604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:16.825820 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:16.825929 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:16.826490 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 02:27:16.826637 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:27:16.827708 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:16.812557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:34731)
0807 02:27:16.813067 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:16.813112 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:16.814515 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:16.815223 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:16.815230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:16.815724 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:16.816379 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:16.816405 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.817509 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.817514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:16.819018 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:16.819035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:16.819701 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:16.819708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:16.819962 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:16.820725 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:16.820743 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:16.822497 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:16.824624 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:16.824631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:16.824649 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:16.825023 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:16.825327 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:16.825331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:16.825333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:16.825488 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:16.825933 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:16.825942 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:16.826227 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 02:27:16.826430 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 02:27:16.826811 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":174,"threads_started":1}
WARNING: no policy specified for oryx/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:27:16.829106 27082 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:16.829419 27082 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:16.829975 27082 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.2@KRBTEST.COM" created.
I20260807 02:27:16.831771 26797 ts_manager.cc:194] Registered new tserver with Master: 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661)
I20260807 02:27:16.833207 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:34731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
I20260807 02:27:16.848269 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:16.970527 27091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:16.970791 27091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:16.970835 27091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:16.975015 27091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:16.975076 27091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:16.975164 27091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:16.981127 27091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:16.982267 27091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:16.983412 27091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:16.993224 27104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:16.993242 27098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:16.993427 27091 server_base.cc:1034] running on GCE node
W20260807 02:27:16.993232 27106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:16.993917 27091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:16.994616 27091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:16.995810 27091 hybrid_clock.cc:648] HybridClock initialized: now 1786069636995790 us; error 33 us; skew 500 ppm
Aug 07 02:27:16 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:16.998737 27091 init.cc:377] Logged in from keytab as oryx/127.17.53.2@KRBTEST.COM (short username oryx)
I20260807 02:27:16.999708 27091 webserver.cc:460] Webserver started at http://127.17.53.2:46861/ using document root <none> and password file <none>
I20260807 02:27:17.000360 27091 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.000463 27091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.000715 27091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:17.002924 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "51e05f0338254057b9cc200b6748afa3"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "00de386eaea569190016f6ded4f1d39d"
server_key_iv: "cb6cc3252161ed5c56ebf04259032e16"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.003526 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "51e05f0338254057b9cc200b6748afa3"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "00de386eaea569190016f6ded4f1d39d"
server_key_iv: "cb6cc3252161ed5c56ebf04259032e16"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.007606 27091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:27:17.010370 27123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.011773 27091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:27:17.011933 27091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "51e05f0338254057b9cc200b6748afa3"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "00de386eaea569190016f6ded4f1d39d"
server_key_iv: "cb6cc3252161ed5c56ebf04259032e16"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.012068 27091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.034330 27091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:17.038020 27091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:17.038241 27091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:17.038902 27091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:17.039788 27091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:17.039860 27091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.039932 27091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:17.039983 27091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.049510 27091 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:34243
I20260807 02:27:17.049541 27292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:34243 every 8 connection(s)
I20260807 02:27:17.050439 27091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:17.056622 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 27091
I20260807 02:27:17.056813 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:17.057152 17620 external_mini_cluster.cc:1428] Setting key 2af41244848f43332a3cdcf4fedbf9b7
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:17.066943 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.052569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:35239 (local address 127.17.53.62:40831)
0807 02:27:17.052715 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.052719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.053721 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 02:27:17.055389 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.055393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.055458 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.055571 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.056936 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.057641 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.059269 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.059561 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.059709 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.062992 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.062999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.063005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.063039 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.064815 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.065343 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.065347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.065349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.065435 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.065810 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.065814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.065816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.066011 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.066120 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.066645 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.066771 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:27:17.067849 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.053019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:35239)
0807 02:27:17.053542 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.053589 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.055136 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.055772 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.055779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.056171 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.056699 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.056726 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.057856 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.057860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.059098 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.059111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.059758 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.059765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.060052 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.060807 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.060834 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.062836 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.064937 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.064944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.064960 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.065228 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.065541 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.065544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.065547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.065694 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.066121 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.066129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.066392 (+ 263us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.066583 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.066928 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for oryx/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:27:17.069078 27293 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:17.069341 27293 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:17.069862 27293 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
I20260807 02:27:17.070960 26797 ts_manager.cc:194] Registered new tserver with Master: 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.3@KRBTEST.COM" created.
I20260807 02:27:17.071646 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:35239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
I20260807 02:27:17.090031 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:17.204676 27302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:17.205008 27302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:17.205060 27302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:17.208935 27302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:17.208981 27302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:17.209036 27302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:17.214046 27302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:17.215063 27302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:17.216039 27302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:17.226402 27314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.226488 27313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.226523 27317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:17.226737 27302 server_base.cc:1034] running on GCE node
I20260807 02:27:17.227247 27302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:17.227985 27302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:17.229175 27302 hybrid_clock.cc:648] HybridClock initialized: now 1786069637229140 us; error 46 us; skew 500 ppm
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:17.232074 27302 init.cc:377] Logged in from keytab as oryx/127.17.53.3@KRBTEST.COM (short username oryx)
I20260807 02:27:17.233073 27302 webserver.cc:460] Webserver started at http://127.17.53.3:39009/ using document root <none> and password file <none>
I20260807 02:27:17.233667 27302 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.233768 27302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.234026 27302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:17.236145 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "9463ecf8e1df4087a0f0cc9db9055659"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "84a898173561c56eea09ba2402f52492"
server_key_iv: "672dbd37419509b1cea2f0fa7162b9ec"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.236853 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "9463ecf8e1df4087a0f0cc9db9055659"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "84a898173561c56eea09ba2402f52492"
server_key_iv: "672dbd37419509b1cea2f0fa7162b9ec"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.240840 27302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:17.243448 27328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.244901 27302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:17.245038 27302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "9463ecf8e1df4087a0f0cc9db9055659"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "84a898173561c56eea09ba2402f52492"
server_key_iv: "672dbd37419509b1cea2f0fa7162b9ec"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.245147 27302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.268877 27302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:17.272066 27302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:17.272300 27302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:17.272950 27302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:17.273813 27302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:17.273895 27302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.273988 27302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:17.274042 27302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.283305 27302 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38235
I20260807 02:27:17.283332 27496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38235 every 8 connection(s)
I20260807 02:27:17.284209 27302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:17.287526 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 27302
I20260807 02:27:17.287633 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:17.287936 17620 external_mini_cluster.cc:1428] Setting key ae82b23d1f4bef44c023900e28df0eb8
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:17.299865 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.286092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34795 (local address 127.17.53.62:40831)
0807 02:27:17.286253 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.286257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.287271 (+ 1014us) server_negotiation.cc:407] Connection header received
0807 02:27:17.289229 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.289233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.289299 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.289424 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.290852 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.291461 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.292819 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.293180 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.293317 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.296132 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.296136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.296138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.296165 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.297744 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.298284 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.298288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.298290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.298387 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.298737 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.298741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.298743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.298933 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.299044 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.299547 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.299684 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 02:27:17.300680 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.286455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:34795)
0807 02:27:17.286967 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.287016 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.288971 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.289629 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.289636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.290102 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.290685 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.290701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.291625 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.291629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.292621 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.292633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.293360 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.293365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.293562 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.294471 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.294492 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.295964 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.297878 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.297884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.297898 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.298170 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.298482 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.298485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.298488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.298637 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.299044 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.299053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.299305 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.299503 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.299830 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260807 02:27:17.301966 27501 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:17.302217 27501 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:17.302733 27501 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
I20260807 02:27:17.303768 26797 ts_manager.cc:194] Registered new tserver with Master: 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:17.304327 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:34795
I20260807 02:27:17.314203 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.53.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.61.keytab.
Entry for principal oryx/127.17.53.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.61.keytab.
I20260807 02:27:17.343902 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:38953
--webserver_interface=127.17.53.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.61.keytab
--principal=oryx/127.17.53.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:40831,127.17.53.61:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:17.471799 27507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:17.472054 27507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:17.472093 27507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:17.476547 27507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:17.476635 27507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:17.476658 27507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:17.476678 27507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:17.476696 27507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:17.483364 27507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:40831,127.17.53.61:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.61.keytab
--principal=oryx/127.17.53.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.53.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
W20260807 02:27:17.483592 27507 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.53.62:40831,127.17.53.61:38953'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 02:27:17.484532 27507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:17.485898 27507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:17.495561 27515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.495561 27517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.495570 27520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:17.495683 27507 server_base.cc:1034] running on GCE node
I20260807 02:27:17.496476 27507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:17.497700 27507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:17.498916 27507 hybrid_clock.cc:648] HybridClock initialized: now 1786069637498889 us; error 46 us; skew 500 ppm
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:17.502063 27507 init.cc:377] Logged in from keytab as oryx/127.17.53.61@KRBTEST.COM (short username oryx)
I20260807 02:27:17.502956 27507 webserver.cc:460] Webserver started at http://127.17.53.61:38825/ using document root <none> and password file <none>
I20260807 02:27:17.503553 27507 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.503650 27507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.503901 27507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:17.506053 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/instance:
uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "d747eb0411e77081a2f34ce64edc2252"
server_key_iv: "c3fb029ab90ec08ec67f2091d906eb1d"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.506657 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal/instance:
uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "d747eb0411e77081a2f34ce64edc2252"
server_key_iv: "c3fb029ab90ec08ec67f2091d906eb1d"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.510903 27507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:17.513747 27529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.515301 27507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:17.515462 27507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "d747eb0411e77081a2f34ce64edc2252"
server_key_iv: "c3fb029ab90ec08ec67f2091d906eb1d"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.515585 27507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.535444 27507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:17.539363 27507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:17.539599 27507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:17.547482 27507 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.61:38953
I20260807 02:27:17.547768 27599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.61:38953 every 8 connection(s)
I20260807 02:27:17.548753 27507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
I20260807 02:27:17.551622 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 27507
I20260807 02:27:17.551909 27600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:17.557150 27600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.61@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:17.567447 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.553283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.17.53.61:38953)
0807 02:27:17.553696 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.553715 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.553756 (+ 41us) server_negotiation.cc:407] Connection header received
0807 02:27:17.554793 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.554818 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.555240 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.555612 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.556293 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.557772 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.558682 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.559385 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.559578 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.562448 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.562457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.562471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.562497 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.564557 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.565076 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.565083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.565090 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.565254 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.565554 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.565558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.565559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.565921 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.566139 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.566429 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.566720 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260807 02:27:17.569170 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.558944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45814)
0807 02:27:17.559317 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.559355 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.559513 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.560019 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.560023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.560515 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.560728 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.560740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.561573 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.561576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.562306 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.562316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.562850 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.562852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.563020 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.563573 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.563591 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.565422 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.567330 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.567340 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.567346 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.567696 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.568005 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.568008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.568010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.568116 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.568556 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.568564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.568688 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.568885 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.569035 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":140,"threads_started":1}
I20260807 02:27:17.569211 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.559030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.17.53.62:40831)
0807 02:27:17.559163 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.559168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.559444 (+ 276us) server_negotiation.cc:407] Connection header received
0807 02:27:17.559642 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.559646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.559703 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.559810 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.560904 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.561409 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.562465 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.562684 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.562834 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.565569 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.565572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.565574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.565598 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.567188 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.567819 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.567822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.567823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.567902 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.568215 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.568217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.568218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.568419 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.568499 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.568944 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.569066 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:27:17.570587 27600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.61@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
I20260807 02:27:17.578513 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.570997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36634)
0807 02:27:17.571129 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.571143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.571223 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.571464 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.571467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.571619 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.571693 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.571698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.572397 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.572400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.572949 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.572957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.573371 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.573375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.573463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.573847 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.573858 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.575561 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.577255 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.577257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.577258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.577435 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.577662 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.577664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.577665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.577733 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.578152 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.578154 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.578226 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.578310 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.578412 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 02:27:17.578830 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.571046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.17.53.61:38953)
0807 02:27:17.571138 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.571143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.571185 (+ 42us) server_negotiation.cc:407] Connection header received
0807 02:27:17.571322 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.571325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.571364 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.571452 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.571827 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.572285 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.573056 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.573268 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.573370 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.575695 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.575699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.575701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.575723 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.577162 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.577515 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.577517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.577518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.577586 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.577863 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.577867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.577868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.578042 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.578158 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.578416 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.578508 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":43}
I20260807 02:27:17.581624 27600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Bootstrap starting.
I20260807 02:27:17.583922 27600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:17.584594 27600 log.cc:824] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:17.586454 27600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: No bootstrap required, opened a new log
I20260807 02:27:17.589543 27600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } }
I20260807 02:27:17.589735 27600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:17.589803 27600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c5231c4d71404a9d28a8fdbae0fc55, State: Initialized, Role: FOLLOWER
I20260807 02:27:17.590162 27600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } }
I20260807 02:27:17.590607 27609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } } }
I20260807 02:27:17.590715 27609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:17.591324 27600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 02:27:17.595991 27626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:17.596072 27626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:17.597848 27507 master_runner.cc:428] Detected that this master 68c5231c4d71404a9d28a8fdbae0fc55 is joining an existing cluster
I20260807 02:27:17.597915 27507 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.53.61:38953
I20260807 02:27:17.608862 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.600838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.17.53.61:38953)
0807 02:27:17.600950 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.600958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.601208 (+ 250us) server_negotiation.cc:407] Connection header received
0807 02:27:17.601351 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.601354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.601393 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.601490 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.601957 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.602475 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.603423 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.603686 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.603803 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.605058 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.605065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.605068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.605095 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.606908 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607392 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.607399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.607401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.607486 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607830 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.607834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.607836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.608080 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.608192 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.608490 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.608656 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":49}
I20260807 02:27:17.608876 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.600744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36644)
0807 02:27:17.601102 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.601114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.601240 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.601503 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.601506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.601732 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.601833 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.601838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.602601 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.602606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.603296 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.603305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.603811 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.603815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.603912 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.604449 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.604464 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.604907 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.607051 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.607057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.607285 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.607600 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.607606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.607697 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.608199 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.608206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.608310 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.608440 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.608597 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":50,"thread_start_us":79,"threads_started":1}
I20260807 02:27:17.609243 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.600308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.17.53.62:40831)
0807 02:27:17.600446 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.600452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.600623 (+ 171us) server_negotiation.cc:407] Connection header received
0807 02:27:17.600800 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.600804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.600858 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.600913 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.601569 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.602397 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.603711 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.604031 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.604155 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.605478 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.605482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.605485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.605514 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.607296 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607790 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.607797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.607802 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.607903 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.608266 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.608269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.608271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.608491 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.608593 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.608967 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.609083 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:27:17.609725 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.600183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45830)
0807 02:27:17.600516 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.600533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.600637 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.601067 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.601070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.601296 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.601428 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.601436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.602544 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.602548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.603576 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.603589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.604168 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.604172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.604287 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.604812 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.604828 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.605327 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.607427 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.607430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.607433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.607678 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.608002 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.608007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.608009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.608102 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.608638 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.608641 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.608731 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.609423 (+ 692us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.609558 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":33,"thread_start_us":101,"threads_started":1}
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.62@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
I20260807 02:27:17.623801 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.613985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36658)
0807 02:27:17.614378 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.614432 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.614565 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.614861 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.614868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.615297 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.615421 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.615429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.616170 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.616173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.617265 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.617277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.617690 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.617694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.617818 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.618295 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.618310 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.619979 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.621886 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.621897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.621904 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.622142 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.622431 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.622434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.622436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.622553 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.623043 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.623051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.623171 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.623408 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.623615 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
I20260807 02:27:17.623801 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.614021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.17.53.61:38953)
0807 02:27:17.614149 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.614153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.614516 (+ 363us) server_negotiation.cc:407] Connection header received
0807 02:27:17.614690 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.614692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.614733 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.614808 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.615531 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.616060 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.617383 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.617580 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.617661 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.620112 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.620117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.620120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.620148 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.621770 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.622248 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.622253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.622254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.622338 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.622652 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.622656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.622657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.622921 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.623030 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.623478 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.623663 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:27:17.625617 26797 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.53.61:38953: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
member_type: NON_VOTER
last_known_addr {
host: "127.17.53.61"
port: 38953
}
attrs {
promote: true
}
}
dest_uuid: "dd0776d83279461e9688ce1ea8d11ae1"
cas_config_opid_index: -1
I20260807 02:27:17.626344 26797 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } }
I20260807 02:27:17.628290 27642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } }
I20260807 02:27:17.628875 27567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:17.629961 27642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.53.61:38953
I20260807 02:27:17.630057 27567 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Refusing update from remote peer dd0776d83279461e9688ce1ea8d11ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 02:27:17.630278 27642 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } }
I20260807 02:27:17.630373 27642 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:17.630386 27507 master.cc:537] Master@127.17.53.61:38953 shutting down...
I20260807 02:27:17.630712 27641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 02:27:17.633349 26780 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 02:27:17.635100 26780 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 -> Peer 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61:38953): Couldn't send request to peer 68c5231c4d71404a9d28a8fdbae0fc55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 02:27:17.635697 27507 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Raft consensus shutting down.
WARNING: no policy specified for oryx/127.17.53.60@KRBTEST.COM; defaulting to no policy
I20260807 02:27:17.636001 27507 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:17.636128 27507 tablet_replica.cc:330] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.60@KRBTEST.COM" created.
I20260807 02:27:17.641186 27507 master.cc:552] Master@127.17.53.61:38953 shutdown complete.
I20260807 02:27:17.641247 27507 master_runner.cc:305] Clearing existing system tablet
I20260807 02:27:17.641532 27507 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.641620 27507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.642693 27507 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 02:27:17.643580 27650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.643788 27507 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.643944 27507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:17.643981 27507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "d747eb0411e77081a2f34ce64edc2252"
server_key_iv: "c3fb029ab90ec08ec67f2091d906eb1d"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.646200 27507 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:17.648326 27507 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 02:27:17.648448 27507 log.cc:1196] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 02:27:17.648852 27507 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Deleting consensus metadata
I20260807 02:27:17.649082 27507 master_runner.cc:315] Copying system tablet from 127.17.53.62:40831
I20260807 02:27:17.650396 27507 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: tablet copy: Beginning tablet copy session from remote peer at address 127.17.53.62:40831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.60.keytab.
Entry for principal oryx/127.17.53.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.60.keytab.
I20260807 02:27:17.658514 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:45675
--webserver_interface=127.17.53.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.60.keytab
--principal=oryx/127.17.53.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:27:17.660084 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.651160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.17.53.62:40831)
0807 02:27:17.651367 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.651372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.651632 (+ 260us) server_negotiation.cc:407] Connection header received
0807 02:27:17.651763 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.651766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.651819 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.651948 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.652532 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.653316 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.654408 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.654745 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.654883 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.656145 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.656149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.656152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.656193 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.658112 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.658625 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.658632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.658635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.658748 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.659128 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.659132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.659134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.659348 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.659472 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.659777 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.659904 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 02:27:17.660104 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.651013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45834)
0807 02:27:17.651511 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.651525 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.651650 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.651965 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.651969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.652266 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.652396 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.652403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.653513 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.653517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.654263 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.654273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.654886 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.654891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.654982 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.655605 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.655619 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.655975 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.658246 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.658252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.658254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.658501 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.658933 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.658935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.658937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.659017 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.659464 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.659467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.659572 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.659712 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.659873 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":107,"threads_started":1}
I20260807 02:27:17.661094 26822 tablet_copy_service.cc:140] P dd0776d83279461e9688ce1ea8d11ae1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 68c5231c4d71404a9d28a8fdbae0fc55 ({username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:45834)
I20260807 02:27:17.661312 26822 tablet_copy_service.cc:161] P dd0776d83279461e9688ce1ea8d11ae1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 68c5231c4d71404a9d28a8fdbae0fc55 at {username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:45834: session id = 68c5231c4d71404a9d28a8fdbae0fc55-00000000000000000000000000000000
I20260807 02:27:17.663693 26822 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 02:27:17.666328 27507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 02:27:17.668277 27507 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: tablet copy: Starting download of 0 data blocks...
I20260807 02:27:17.668498 27507 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: tablet copy: Starting download of 1 WAL segments...
I20260807 02:27:17.670328 27507 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 02:27:17.671955 26822 tablet_copy_service.cc:342] P dd0776d83279461e9688ce1ea8d11ae1: Request end of tablet copy session 68c5231c4d71404a9d28a8fdbae0fc55-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:45834
I20260807 02:27:17.672083 26822 tablet_copy_service.cc:433] P dd0776d83279461e9688ce1ea8d11ae1: ending tablet copy session 68c5231c4d71404a9d28a8fdbae0fc55-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 68c5231c4d71404a9d28a8fdbae0fc55
I20260807 02:27:17.678902 27507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:17.680627 27671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.680666 27670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.680959 27675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:17.681180 27507 server_base.cc:1034] running on GCE node
I20260807 02:27:17.681418 27507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:17.681722 27507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:17.682857 27507 hybrid_clock.cc:648] HybridClock initialized: now 1786069637682853 us; error 53 us; skew 500 ppm
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:17.684397 27507 init.cc:377] Logged in from keytab as oryx/127.17.53.61@KRBTEST.COM (short username oryx)
I20260807 02:27:17.684929 27507 webserver.cc:460] Webserver started at http://127.17.53.61:34795/ using document root <none> and password file <none>
I20260807 02:27:17.685125 27507 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.685182 27507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.686273 27507 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:17.687215 27687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.687448 27507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:17.687543 27507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "d747eb0411e77081a2f34ce64edc2252"
server_key_iv: "c3fb029ab90ec08ec67f2091d906eb1d"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.687626 27507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.700028 27507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:17.702003 27507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:17.707036 27507 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.61:38953
I20260807 02:27:17.707576 27507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
I20260807 02:27:17.707938 27773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.61:38953 every 8 connection(s)
I20260807 02:27:17.709417 27774 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:27:17.710249 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Bootstrap starting.
I20260807 02:27:17.718649 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:27:17.718940 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Bootstrap complete.
I20260807 02:27:17.719414 27774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } }
I20260807 02:27:17.719547 27774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 68c5231c4d71404a9d28a8fdbae0fc55, State: Initialized, Role: LEARNER
I20260807 02:27:17.719663 27774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } }
I20260807 02:27:17.719981 27779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } }
I20260807 02:27:17.720149 27774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:17.720155 27779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: LEARNER
I20260807 02:27:17.723373 27796 catalog_manager.cc:1259] Loaded cluster ID: 2b996ed1245646dd86900b0d1d784300
I20260807 02:27:17.723522 27796 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: loading cluster ID for follower catalog manager: success
I20260807 02:27:17.725438 27796 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: acquiring CA information for follower catalog manager: success
I20260807 02:27:17.726540 27796 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 02:27:17.821923 27669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:17.822189 27669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:17.822239 27669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:17.826493 27669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:17.826555 27669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:17.826574 27669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:17.826591 27669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:17.826607 27669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:17.831904 27669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.60.keytab
--principal=oryx/127.17.53.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:45675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.53.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:17.832940 27669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:17.833937 27669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:27:17.836323 27082 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
W20260807 02:27:17.843079 27807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.843044 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:17.843091 27805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:17.843238 27669 server_base.cc:1034] running on GCE node
I20260807 02:27:17.843895 27669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:17.845130 27669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:17.846312 27669 hybrid_clock.cc:648] HybridClock initialized: now 1786069637846290 us; error 34 us; skew 500 ppm
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:17.849402 27669 init.cc:377] Logged in from keytab as oryx/127.17.53.60@KRBTEST.COM (short username oryx)
I20260807 02:27:17.850394 27669 webserver.cc:460] Webserver started at http://127.17.53.60:33791/ using document root <none> and password file <none>
I20260807 02:27:17.850979 27669 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:17.851069 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:17.851328 27669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:17.853575 27669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/instance:
uuid: "7e38c210c8d84e18b599671cd60c0ef4"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "b148133c182e4af2eec24a9d9d331c2c"
server_key_iv: "fabcbf04da33a774052c64c488baba50"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.854168 27669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal/instance:
uuid: "7e38c210c8d84e18b599671cd60c0ef4"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "b148133c182e4af2eec24a9d9d331c2c"
server_key_iv: "fabcbf04da33a774052c64c488baba50"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.858047 27669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 02:27:17.860569 27818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:17.862157 27669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:17.862339 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "7e38c210c8d84e18b599671cd60c0ef4"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "b148133c182e4af2eec24a9d9d331c2c"
server_key_iv: "fabcbf04da33a774052c64c488baba50"
server_key_version: "encryptionkey@0"
I20260807 02:27:17.862471 27669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:17.895560 27669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:17.899334 27669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:17.899592 27669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:17.907382 27669 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.60:45675
I20260807 02:27:17.907678 27892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.60:45675 every 8 connection(s)
I20260807 02:27:17.908529 27669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
I20260807 02:27:17.911679 27893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:17.916183 27893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:27:17.918176 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 27669
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.60@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
I20260807 02:27:17.931329 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.918358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.17.53.62:40831)
0807 02:27:17.918560 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.918565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.919363 (+ 798us) server_negotiation.cc:407] Connection header received
0807 02:27:17.920534 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.920538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.920600 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.920701 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.922109 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.922686 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.923842 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.924176 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.924296 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.926952 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.926956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.926958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.926993 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.928720 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.929365 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.929368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.929370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.929450 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.929867 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.929873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.929879 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.930142 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.930252 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.930961 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.931117 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:27:17.932265 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.918697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45840)
0807 02:27:17.919208 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.919261 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.920266 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.920757 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.920835 (+ 78us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.921300 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.921957 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.921972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.922832 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.922837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.923708 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.923718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.924339 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.924344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.924483 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.925219 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.925235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.926788 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.928907 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.928914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.928925 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.929239 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.929550 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.929554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.929555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.929743 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.930261 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.930268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.930609 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.930862 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.931209 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":1,"thread_start_us":151,"threads_started":1}
I20260807 02:27:17.932281 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.919106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.17.53.60:45675)
0807 02:27:17.919662 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.919679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.919725 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:27:17.920269 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.920299 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.920553 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.921408 (+ 855us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.921749 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.922841 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.923768 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.924239 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.924424 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.927483 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.927491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.927506 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.927540 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.929559 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.930071 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.930076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.930077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.930153 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.930463 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.930466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.930468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.930749 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.930952 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.931250 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.931389 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":423,"thread_start_us":171,"threads_started":1}
I20260807 02:27:17.934199 27893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.60@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
I20260807 02:27:17.943028 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.934814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36666)
0807 02:27:17.935019 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.935034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.935116 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.935633 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.935635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.935779 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.935878 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.935882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.936828 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.936832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.937379 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.937387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.937850 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.937854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.937938 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.938380 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.938392 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.939710 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.941640 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.941643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.941644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.941840 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.942137 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.942140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.942142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.942224 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.942596 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.942598 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.942679 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.942799 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.942906 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260807 02:27:17.943106 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.935003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.17.53.61:38953)
0807 02:27:17.935425 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.935429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.935441 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:27:17.935478 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.935481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.935519 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.935600 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.936014 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.936668 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.937486 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.937722 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.937808 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.940046 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.940049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.940051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.940075 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.941531 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.941931 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.941933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.941935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.942018 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.942321 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.942324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.942325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.942507 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.942585 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.942859 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.942956 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":134,"threads_started":1}
I20260807 02:27:17.943790 27893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:27:17 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069637, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.60@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
I20260807 02:27:17.952668 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.944318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58636)
0807 02:27:17.944550 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.944567 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.944657 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.944925 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.944927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.945094 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.945156 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.945160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.945911 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.945917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.946414 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.946421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.946876 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.946879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.946967 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.947384 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.947395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.949312 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.951047 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.951049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.951051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.951261 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.951561 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.951564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.951565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:17.951660 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:17.952085 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:17.952088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:17.952187 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 02:27:17.952305 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 02:27:17.952442 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":7}
I20260807 02:27:17.952675 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.944374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.17.53.60:45675)
0807 02:27:17.944550 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.944556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.944617 (+ 61us) server_negotiation.cc:407] Connection header received
0807 02:27:17.944744 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.944746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.944832 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.944914 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.945274 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.945794 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.946530 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.946771 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.946878 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.949471 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.949475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.949477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.949502 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.950932 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.951361 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.951364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.951365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.951453 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:17.951769 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:17.951773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:17.951774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:17.951952 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:17.952054 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:17.952362 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 02:27:17.952500 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":95}
I20260807 02:27:17.956347 27893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Bootstrap starting.
I20260807 02:27:17.958817 27893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:17.959616 27893 log.cc:824] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:17.961743 27893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: No bootstrap required, opened a new log
I20260807 02:27:17.964918 27893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } }
I20260807 02:27:17.965142 27893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:17.965176 27893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e38c210c8d84e18b599671cd60c0ef4, State: Initialized, Role: FOLLOWER
I20260807 02:27:17.965749 27893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } }
I20260807 02:27:17.966295 27905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } } }
I20260807 02:27:17.966447 27905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:17.967181 27893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 02:27:17.972826 27919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:17.972941 27919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:17.976928 27669 master_runner.cc:428] Detected that this master 7e38c210c8d84e18b599671cd60c0ef4 is joining an existing cluster
I20260807 02:27:17.977007 27669 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.53.60:45675
I20260807 02:27:17.986533 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 26882
I20260807 02:27:18.012005 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.979866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.17.53.62:40831)
0807 02:27:17.980009 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.980013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.980618 (+ 605us) server_negotiation.cc:407] Connection header received
0807 02:27:17.980792 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.980798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.980858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.980950 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.981528 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.982073 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.983050 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.983329 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.983449 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.984637 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:17.984641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:17.984644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:17.984673 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:17.989072 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.010137 (+ 21065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.010143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.010146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.010285 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.010698 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.010705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.010711 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.011031 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.011143 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.011616 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.011784 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:27:18.012676 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.979748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45856)
0807 02:27:17.980513 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.980526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.980654 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.980971 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.980973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.981262 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.981400 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.981406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.982235 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.982238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.982917 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.982927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.983462 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.983466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.983558 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.984108 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.984129 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:17.984489 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:17.989209 (+ 4720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:17.989215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:17.989218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.009960 (+ 20742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.010442 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.010448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.010453 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.010560 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.011230 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.011277 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.011406 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.012291 (+ 885us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.012475 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":126,"threads_started":1}
I20260807 02:27:18.013079 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.980117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58644)
0807 02:27:17.980620 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.980635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.981096 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.981766 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.981771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.982009 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.982128 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.982135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.983778 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.983782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.984703 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.984715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.985471 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.985474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.985575 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:17.989309 (+ 3734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:17.989322 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.009299 (+ 19977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.011526 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.011532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.011537 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.011770 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.012028 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.012032 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.012034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.012116 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.012539 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.012541 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.012617 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.012701 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.012921 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":132,"threads_started":1}
I20260807 02:27:18.013602 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.979864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.17.53.60:45675)
0807 02:27:17.980012 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.980016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.980710 (+ 694us) server_negotiation.cc:407] Connection header received
0807 02:27:17.981304 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.981307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.981347 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.981394 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.982857 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.983626 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.984903 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.985261 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.985334 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.009519 (+ 24185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.009524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.009526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.009554 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.011408 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.011870 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.011873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.011875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.011944 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.012206 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.012208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.012210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.012412 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.012505 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.013374 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.013483 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 02:27:18.019129 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.980356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.17.53.61:38953)
0807 02:27:17.980475 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:17.980480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:17.981021 (+ 541us) server_negotiation.cc:407] Connection header received
0807 02:27:17.981521 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:17.981524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:17.981573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:17.981625 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:17.984875 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.985733 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.986844 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.987256 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.987359 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.014974 (+ 27615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.014981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.014988 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.015030 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.017152 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.017672 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.017675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.017677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.017786 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.018120 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.018123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.018125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.018386 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.018473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.018842 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.018977 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:27:18.019641 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:17.980251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36682)
0807 02:27:17.980907 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:17.980920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:17.981189 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:17.982280 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:17.982282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:17.982549 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:17.982673 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.982681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.985866 (+ 3185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.985871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.986698 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:17.986712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:17.987556 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:17.987562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:17.987683 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.009299 (+ 21616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.009319 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.014780 (+ 5461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.017287 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.017290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.017293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.017542 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.017909 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.017911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.017914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.018006 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.018535 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.018540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.018630 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.019330 (+ 700us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.019472 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":75,"threads_started":1}
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069636, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.62@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
I20260807 02:27:18.025202 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:36661
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:40831
--webserver_port=35539
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:27:18.027704 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.017477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58646)
0807 02:27:18.017757 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.017772 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.017852 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.018341 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.018345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.018565 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.018676 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.018686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.019749 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.019754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.020499 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.020513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.021303 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.021307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.021412 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.021881 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.021910 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.023604 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.025933 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.025937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.025939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.026188 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.026510 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.026513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.026515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.026610 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.027161 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.027164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.027272 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.027388 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.027546 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260807 02:27:18.027704 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.017931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58646 (local address 127.17.53.60:45675)
0807 02:27:18.018053 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.018059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.018072 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:27:18.018139 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.018142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.018198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.018789 (+ 591us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.018886 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.019612 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.020693 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.021166 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.021311 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.023748 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.023759 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.023762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.023788 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.025815 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.026300 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.026306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.026311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.026407 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.026713 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.026727 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.026732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.026995 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.027163 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.027441 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.027544 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 02:27:18.028738 26797 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.53.60:45675: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4"
member_type: NON_VOTER
last_known_addr {
host: "127.17.53.60"
port: 45675
}
attrs {
promote: true
}
}
dest_uuid: "dd0776d83279461e9688ce1ea8d11ae1"
cas_config_opid_index: 5
I20260807 02:27:18.029409 26797 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } }
I20260807 02:27:18.030596 27641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.032017 27641 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.53.60:45675
I20260807 02:27:18.032210 27641 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.032336 27641 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.033111 27669 master.cc:537] Master@127.17.53.60:45675 shutting down...
W20260807 02:27:18.036161 26782 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 -> Peer 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675): Couldn't send request to peer 7e38c210c8d84e18b599671cd60c0ef4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 02:27:18.039531 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.032541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.17.53.61:38953)
0807 02:27:18.032683 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.032688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.033534 (+ 846us) server_negotiation.cc:407] Connection header received
0807 02:27:18.033597 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.033602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.033680 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.033783 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:18.034233 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.035201 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.037272 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.037915 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.038032 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.038246 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.039297 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.039391 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:27:18.039844 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.030739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36690)
0807 02:27:18.033334 (+ 2595us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.033351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.033439 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.033950 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.033956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.033978 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:18.034072 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.034077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.035340 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.035343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.037113 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.037127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.038319 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.038324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.038436 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.038484 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.038560 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.038634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2509}
I20260807 02:27:18.039987 27669 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:18.040298 27669 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:18.040385 27736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 LEARNER]: Refusing update from remote peer dd0776d83279461e9688ce1ea8d11ae1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 02:27:18.040421 27669 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: stopping tablet replica
I20260807 02:27:18.040884 27642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 02:27:18.042909 27736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.045099 27779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd0776d83279461e9688ce1ea8d11ae1. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.045259 27779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: LEARNER
I20260807 02:27:18.045367 27779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: NON_VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: true } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.045482 27779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: LEARNER
I20260807 02:27:18.046578 27642 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: attempting to promote NON_VOTER 68c5231c4d71404a9d28a8fdbae0fc55 to VOTER
I20260807 02:27:18.046981 27642 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } }
I20260807 02:27:18.048272 27736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 LEARNER]: Refusing update from remote peer dd0776d83279461e9688ce1ea8d11ae1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 02:27:18.048355 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.048058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58656)
0807 02:27:18.048200 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.048274 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:45675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 02:27:18.048732 27641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 02:27:18.049405 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.049014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58664)
0807 02:27:18.049191 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.049242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:45675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260807 02:27:18.049533 26782 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 -> Peer 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675): Couldn't send request to peer 7e38c210c8d84e18b599671cd60c0ef4. Status: Network error: Client connection negotiation failed: client connection to 127.17.53.60:45675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 02:27:18.050477 27641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.051718 27641 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.051870 27641 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.052019 27641 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.052104 27641 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.052796 27736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.054056 27939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.054190 27939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:18.056260 27669 master.cc:552] Master@127.17.53.60:45675 shutdown complete.
I20260807 02:27:18.056331 27669 master_runner.cc:305] Clearing existing system tablet
I20260807 02:27:18.056572 27669 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:18.056646 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:18.057777 27669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:18.058640 27946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.058869 27669 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:18.059027 27669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:18.059065 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "7e38c210c8d84e18b599671cd60c0ef4"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "b148133c182e4af2eec24a9d9d331c2c"
server_key_iv: "fabcbf04da33a774052c64c488baba50"
server_key_version: "encryptionkey@0"
I20260807 02:27:18.060740 27669 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:18.062644 27669 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 02:27:18.062753 27669 log.cc:1196] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 02:27:18.062987 27669 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Deleting consensus metadata
I20260807 02:27:18.063174 27669 master_runner.cc:315] Copying system tablet from 127.17.53.62:40831
I20260807 02:27:18.064289 27669 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: tablet copy: Beginning tablet copy session from remote peer at address 127.17.53.62:40831
I20260807 02:27:18.073436 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.064892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:45870)
0807 02:27:18.065209 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.065221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.065345 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.065602 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.065606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.065821 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.065928 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.065934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.066699 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.066705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.067448 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.067458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.068982 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.068986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.069084 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.069671 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.069684 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.070010 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.071807 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.071812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.071814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.072026 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.072311 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.072315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.072317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.072408 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.072848 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.072852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.072957 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.073074 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.073234 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260807 02:27:18.073426 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.064979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.17.53.62:40831)
0807 02:27:18.065105 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.065110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.065299 (+ 189us) server_negotiation.cc:407] Connection header received
0807 02:27:18.065446 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.065450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.065503 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.065585 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.066048 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.066586 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.067571 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.067806 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.069014 (+ 1208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.070153 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.070157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.070159 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.070186 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.071682 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.072128 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.072132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.072135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.072219 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.072501 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.072506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.072507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.072693 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.072879 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.073142 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.073259 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:27:18.073839 27293 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
I20260807 02:27:18.074347 26822 tablet_copy_service.cc:140] P dd0776d83279461e9688ce1ea8d11ae1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e38c210c8d84e18b599671cd60c0ef4 ({username='oryx', principal='oryx/127.17.53.60@KRBTEST.COM'} at 127.0.0.1:45870)
I20260807 02:27:18.074448 26822 tablet_copy_service.cc:161] P dd0776d83279461e9688ce1ea8d11ae1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e38c210c8d84e18b599671cd60c0ef4 at {username='oryx', principal='oryx/127.17.53.60@KRBTEST.COM'} at 127.0.0.1:45870: session id = 7e38c210c8d84e18b599671cd60c0ef4-00000000000000000000000000000000
I20260807 02:27:18.075476 26822 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 02:27:18.076675 27669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 02:27:18.078960 27669 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: tablet copy: Starting download of 0 data blocks...
I20260807 02:27:18.079177 27669 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: tablet copy: Starting download of 1 WAL segments...
I20260807 02:27:18.080543 27669 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 02:27:18.082180 26822 tablet_copy_service.cc:342] P dd0776d83279461e9688ce1ea8d11ae1: Request end of tablet copy session 7e38c210c8d84e18b599671cd60c0ef4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.53.60@KRBTEST.COM'} at 127.0.0.1:45870
I20260807 02:27:18.082275 26822 tablet_copy_service.cc:433] P dd0776d83279461e9688ce1ea8d11ae1: ending tablet copy session 7e38c210c8d84e18b599671cd60c0ef4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e38c210c8d84e18b599671cd60c0ef4
I20260807 02:27:18.089442 27669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:18.091053 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.091097 27956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.091193 27957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:18.091884 27669 server_base.cc:1034] running on GCE node
I20260807 02:27:18.092322 27669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:18.092633 27669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:18.093779 27669 hybrid_clock.cc:648] HybridClock initialized: now 1786069638093773 us; error 30 us; skew 500 ppm
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:18.095284 27669 init.cc:377] Logged in from keytab as oryx/127.17.53.60@KRBTEST.COM (short username oryx)
I20260807 02:27:18.095801 27669 webserver.cc:460] Webserver started at http://127.17.53.60:40913/ using document root <none> and password file <none>
I20260807 02:27:18.096004 27669 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:18.096062 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:18.097235 27669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:18.098196 27968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.098470 27669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:18.098572 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "7e38c210c8d84e18b599671cd60c0ef4"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "b148133c182e4af2eec24a9d9d331c2c"
server_key_iv: "fabcbf04da33a774052c64c488baba50"
server_key_version: "encryptionkey@0"
I20260807 02:27:18.098685 27669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:18.109560 27669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:18.111642 27669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:18.116873 27669 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.60:45675
I20260807 02:27:18.116882 28052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.60:45675 every 8 connection(s)
I20260807 02:27:18.117447 27669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
I20260807 02:27:18.118893 28053 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:27:18.119879 28053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Bootstrap starting.
I20260807 02:27:18.129515 28053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:27:18.129832 28053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: Bootstrap complete.
I20260807 02:27:18.130352 28053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } }
I20260807 02:27:18.130492 28053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e38c210c8d84e18b599671cd60c0ef4, State: Initialized, Role: LEARNER
I20260807 02:27:18.130609 28053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } }
I20260807 02:27:18.130936 28056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.131101 28056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: LEARNER
I20260807 02:27:18.131124 28053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:18.134469 28072 catalog_manager.cc:1259] Loaded cluster ID: 2b996ed1245646dd86900b0d1d784300
I20260807 02:27:18.134570 28072 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: loading cluster ID for follower catalog manager: success
I20260807 02:27:18.136174 28072 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: acquiring CA information for follower catalog manager: success
I20260807 02:27:18.137388 28072 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 02:27:18.188910 27936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:18.189292 27936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:18.189358 27936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:18.195930 27936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:18.196012 27936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:18.196102 27936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:18.203990 27936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:36661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=35539
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:18.205381 27936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:18.206724 27936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:18.217134 28083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.217129 28079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.217154 28078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:18.217553 27936 server_base.cc:1034] running on GCE node
I20260807 02:27:18.218003 27936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:18.218703 27936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:18.219892 27936 hybrid_clock.cc:648] HybridClock initialized: now 1786069638219877 us; error 28 us; skew 500 ppm
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:18.222965 27936 init.cc:377] Logged in from keytab as oryx/127.17.53.1@KRBTEST.COM (short username oryx)
I20260807 02:27:18.224011 27936 webserver.cc:460] Webserver started at http://127.17.53.1:35539/ using document root <none> and password file <none>
I20260807 02:27:18.224689 27936 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:18.224874 27936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:18.229948 27936 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 02:27:18.232380 28095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.233711 27936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:18.233861 27936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "70bf2b351775d4d97490614a4d11a0a4"
server_key_iv: "8fd2defdac12fc97050bd1f87353827d"
server_key_version: "encryptionkey@0"
I20260807 02:27:18.234408 27936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:18.247584 27936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:18.251140 27936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:18.251358 27936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:18.251994 27936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:18.253023 27936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:18.253086 27936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.253145 27936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:18.253180 27936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.262408 27936 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:36661
I20260807 02:27:18.262481 28278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:36661 every 8 connection(s)
I20260807 02:27:18.263463 27936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:18.264158 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 27936
I20260807 02:27:18.264410 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 27091
I20260807 02:27:18.278940 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:34243
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:40831
--webserver_port=46861
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
I20260807 02:27:18.290796 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.276560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49523 (local address 127.17.53.61:38953)
0807 02:27:18.276711 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.276716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.276734 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:27:18.276797 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.276802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.276863 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.276950 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.279582 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.280619 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.281923 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.282278 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.282373 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.286538 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.286542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.286545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.286572 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.288247 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.288899 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.288906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.288909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.288995 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.289422 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.289425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.289427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.289652 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.289749 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.290486 (+ 737us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.290612 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 02:27:18.291332 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.269295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:36435 (local address 127.17.53.62:40831)
0807 02:27:18.269921 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.269928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.271962 (+ 2034us) server_negotiation.cc:407] Connection header received
0807 02:27:18.273519 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.273525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.273596 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.273655 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.275692 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.276568 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.278657 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.279033 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.279172 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.282927 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.282930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.282933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.282964 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.284858 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.289198 (+ 4340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.289201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.289203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.289305 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.289612 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.289615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.289616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.289849 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.289941 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.291062 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.291179 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449}
I20260807 02:27:18.292171 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.271315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.17.53.1:49523)
0807 02:27:18.275948 (+ 4633us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.275965 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.276053 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.278989 (+ 2936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.278993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.279244 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.279418 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.279429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.280881 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.280887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.281769 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.281782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.282566 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.282569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.282664 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.284896 (+ 2232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.284911 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.286383 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.288384 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.288390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.288406 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.288778 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.289115 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.289121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.289124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.289306 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.289820 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.289827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.290169 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.291020 (+ 851us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.291174 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4533,"spinlock_wait_cycles":8576,"thread_start_us":81,"threads_started":1}
I20260807 02:27:18.293092 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.268244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:36435)
0807 02:27:18.271780 (+ 3536us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.271856 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.273253 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.274086 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.274098 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.274656 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.275478 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.275504 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.276684 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.276690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.278301 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.278316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.279557 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.279562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.279794 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.280505 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.280536 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.282703 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.288826 (+ 6123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.288830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.288832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.289081 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.289415 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.289418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.289420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.289504 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.289982 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.289985 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.290169 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.290404 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.290742 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2061,"spinlock_wait_cycles":4992,"thread_start_us":803,"threads_started":1}
I20260807 02:27:18.293589 28282 heartbeater.cc:347] Connected to a master server at 127.17.53.61:38953
I20260807 02:27:18.293951 28282 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.294623 28282 heartbeater.cc:511] Master 127.17.53.61:38953 requested a full tablet report, sending...
I20260807 02:27:18.296489 27724 ts_manager.cc:194] Registered new tserver with Master: 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661)
I20260807 02:27:18.297485 28280 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:18.297569 28280 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.297801 28280 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
I20260807 02:27:18.298697 26797 ts_manager.cc:194] Re-registered known tserver with Master: 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661)
I20260807 02:27:18.299530 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:36435
I20260807 02:27:18.303108 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.270092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.17.53.1:35105)
0807 02:27:18.273567 (+ 3475us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.273585 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.273694 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.280157 (+ 6463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.280161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.282746 (+ 2585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.282973 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.282987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.284670 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.284676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.285815 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.285832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.286632 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.286636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.286735 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.293933 (+ 7198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.293947 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.298918 (+ 4971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.301329 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.301333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.301335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.301592 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.301937 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.301940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.301942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.302034 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.302534 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.302538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.302644 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.302741 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.302921 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3389,"thread_start_us":230,"threads_started":1}
I20260807 02:27:18.303675 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.269421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35105 (local address 127.17.53.60:45675)
0807 02:27:18.276291 (+ 6870us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.276298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.276315 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:18.276363 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.276369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.276427 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.278829 (+ 2402us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.283277 (+ 4448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.284513 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.285967 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.286362 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.286482 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.299110 (+ 12628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.299115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.299117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.299151 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.301193 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.301731 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.301738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.301742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.301839 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.302144 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.302149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.302150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.302401 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.302505 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.303056 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.303472 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6762,"thread_start_us":154,"threads_started":1}
I20260807 02:27:18.304543 28279 heartbeater.cc:347] Connected to a master server at 127.17.53.60:45675
I20260807 02:27:18.304665 28279 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.304988 28279 heartbeater.cc:511] Master 127.17.53.60:45675 requested a full tablet report, sending...
I20260807 02:27:18.306486 27994 ts_manager.cc:194] Registered new tserver with Master: 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661)
I20260807 02:27:18.306437 27501 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
W20260807 02:27:18.425566 28290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:18.425861 28290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:18.425913 28290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:18.430429 28290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:18.430490 28290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:18.430594 28290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:18.436607 28290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:34243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=46861
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:18.437789 28290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:18.438967 28290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:18.449468 28296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.449539 28300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.449469 28297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:18.449950 28290 server_base.cc:1034] running on GCE node
I20260807 02:27:18.450318 28290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:18.450955 28290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:18.452142 28290 hybrid_clock.cc:648] HybridClock initialized: now 1786069638452103 us; error 46 us; skew 500 ppm
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:18.455189 28290 init.cc:377] Logged in from keytab as oryx/127.17.53.2@KRBTEST.COM (short username oryx)
I20260807 02:27:18.456140 28290 webserver.cc:460] Webserver started at http://127.17.53.2:46861/ using document root <none> and password file <none>
I20260807 02:27:18.456741 28290 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:18.456918 28290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:18.460907 28290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:18.463165 28312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.464506 28290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:18.464650 28290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "51e05f0338254057b9cc200b6748afa3"
format_stamp: "Formatted at 2026-08-07 02:27:16 on dist-test-slave-g170"
server_key: "00de386eaea569190016f6ded4f1d39d"
server_key_iv: "cb6cc3252161ed5c56ebf04259032e16"
server_key_version: "encryptionkey@0"
I20260807 02:27:18.465296 28290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:18.486384 28290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:18.489495 28290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:18.489691 28290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:18.490283 28290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:18.491132 28290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:18.491199 28290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.491273 28290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:18.491320 28290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.501802 28290 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:34243
I20260807 02:27:18.501863 28509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:34243 every 8 connection(s)
I20260807 02:27:18.502821 28290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:18.508646 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 28290
I20260807 02:27:18.508930 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 27302
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
I20260807 02:27:18.525559 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.504735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:51385 (local address 127.17.53.61:38953)
0807 02:27:18.504907 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.504915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.506471 (+ 1556us) server_negotiation.cc:407] Connection header received
0807 02:27:18.507702 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.507705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.507760 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.507817 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.512285 (+ 4468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.513120 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515107 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.515449 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515524 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.520077 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.520081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.520084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.520115 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.522725 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523410 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.523414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.523416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.523514 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523985 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.523989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.523991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.524231 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.524341 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.525229 (+ 888us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.525375 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 02:27:18.525559 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.506980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:33587 (local address 127.17.53.62:40831)
0807 02:27:18.507107 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.507111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.507225 (+ 114us) server_negotiation.cc:407] Connection header received
0807 02:27:18.507702 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.507705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.507760 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.507848 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.512278 (+ 4430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.513119 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515106 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.515448 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515524 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.520085 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.520089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.520091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.520120 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.522390 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523410 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.523414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.523416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.523514 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523985 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.523989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.523991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.524231 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.524318 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.525229 (+ 911us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.525375 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:18.527407 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.505265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.17.53.2:51385)
0807 02:27:18.506319 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.506360 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.507456 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.508236 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.508245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.508882 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.512093 (+ 3211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.512117 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.513302 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.513306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.514963 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.514975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.515689 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.516015 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.516919 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.516938 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.519843 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.522878 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.522881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.522883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.523281 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.523646 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.523655 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.523858 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.524419 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.524426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.524884 (+ 458us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.526264 (+ 1380us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.526407 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":332,"spinlock_wait_cycles":12672,"thread_start_us":689,"threads_started":1}
I20260807 02:27:18.527407 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.506859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:33587)
0807 02:27:18.507137 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.507150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.507456 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.508236 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.508245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.508882 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.512028 (+ 3146us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x6618de8. stack: 00007f950338dc78 00007f950338e1a9 00007f9502f501d7 00000000005984a6 00000000005984f9 000000000059c488 00007f950338a2ab 00007f9503389ad0 00007f950c1610a5 00007f950c16158c 00007f9503e0b1fc 00007f95079e0b1f 00007f9507a21832 00007f9507a2268c 00007f9507a380d3 00007f9507a3b54b
0807 02:27:18.512142 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.512148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.513346 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.513352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.514963 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.514975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.515676 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.515680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.516014 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.516828 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.516862 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.519843 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.522602 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.522612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.522632 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.523273 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.523725 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.523727 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.523729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.523861 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.524419 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.524426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.524884 (+ 458us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.525737 (+ 853us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.526167 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":354,"spinlock_wait_cycles":5507200,"thread_start_us":87,"threads_started":1}
I20260807 02:27:18.529882 28511 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:18.530164 28511 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.530791 28513 heartbeater.cc:347] Connected to a master server at 127.17.53.61:38953
I20260807 02:27:18.530846 28511 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
I20260807 02:27:18.530885 28513 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.531032 28513 heartbeater.cc:511] Master 127.17.53.61:38953 requested a full tablet report, sending...
I20260807 02:27:18.532092 27724 ts_manager.cc:194] Registered new tserver with Master: 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
I20260807 02:27:18.532104 26797 ts_manager.cc:194] Re-registered known tserver with Master: 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
I20260807 02:27:18.533130 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:33587
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
I20260807 02:27:18.537848 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:38235
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:40831
--webserver_port=39009
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:27:18.540400 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.507645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:53185 (local address 127.17.53.60:45675)
0807 02:27:18.507765 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.507769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.507994 (+ 225us) server_negotiation.cc:407] Connection header received
0807 02:27:18.508164 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.508167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.508228 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.508279 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.530178 (+ 21899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.531242 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.533115 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.533494 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.533600 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.536673 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.536677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.536679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.536707 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.538611 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.539142 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.539148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.539153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.539256 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.539531 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.539535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.539537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.539720 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.539848 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.540123 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.540247 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:27:18.540418 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.507295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.17.53.2:53185)
0807 02:27:18.507920 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.507932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.508057 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.529571 (+ 21514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.529577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.529810 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.529969 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.529977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.531480 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.531483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.532983 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.532993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.533631 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.533636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.533730 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.534297 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.534312 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.536519 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.538781 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.538784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.538787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.539043 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.539353 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.539355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.539356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.539431 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.539860 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.539866 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.539958 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.540068 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.540223 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":114,"threads_started":1}
I20260807 02:27:18.541249 28510 heartbeater.cc:347] Connected to a master server at 127.17.53.60:45675
I20260807 02:27:18.541366 28510 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.541579 28510 heartbeater.cc:511] Master 127.17.53.60:45675 requested a full tablet report, sending...
I20260807 02:27:18.542413 27994 ts_manager.cc:194] Registered new tserver with Master: 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
W20260807 02:27:18.654350 28523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:18.654585 28523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:18.654626 28523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:18.658654 28523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:18.658711 28523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:18.658774 28523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:18.663868 28523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:38235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=39009
--tserver_master_addrs=127.17.53.62:40831,127.17.53.61:38953,127.17.53.60:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:18.664901 28523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:18.665887 28523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:27:18.676705 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.672172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.17.53.60:45675)
0807 02:27:18.672320 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.672327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.672629 (+ 302us) server_negotiation.cc:407] Connection header received
0807 02:27:18.672780 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.672784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.672856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.672976 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:18.673252 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.674047 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.675110 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.675840 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.675953 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.676116 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.676487 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.676566 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:27:18.676750 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.672096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58674)
0807 02:27:18.672502 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.672522 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.672659 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.673013 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.673016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.673040 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:18.673138 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.673143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.674158 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.674161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.674969 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.674980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.676100 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.676107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.676234 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.676331 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.676474 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.676571 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":79,"threads_started":1}
W20260807 02:27:18.677001 28530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:18.677353 28533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:18.677647 28010 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
W20260807 02:27:18.677944 28529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:18.678114 28523 server_base.cc:1034] running on GCE node
I20260807 02:27:18.678198 28536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd0776d83279461e9688ce1ea8d11ae1. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.678380 28536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: LEARNER
I20260807 02:27:18.678727 28523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:18.679409 28523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:18.680606 28523 hybrid_clock.cc:648] HybridClock initialized: now 1786069638680599 us; error 65 us; skew 500 ppm
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:18.683414 28523 init.cc:377] Logged in from keytab as oryx/127.17.53.3@KRBTEST.COM (short username oryx)
I20260807 02:27:18.684347 28523 webserver.cc:460] Webserver started at http://127.17.53.3:39009/ using document root <none> and password file <none>
I20260807 02:27:18.684998 28523 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:18.685114 28523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:18.689046 28523 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:18.691087 28543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.692373 28523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:18.692512 28523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "9463ecf8e1df4087a0f0cc9db9055659"
format_stamp: "Formatted at 2026-08-07 02:27:17 on dist-test-slave-g170"
server_key: "84a898173561c56eea09ba2402f52492"
server_key_iv: "672dbd37419509b1cea2f0fa7162b9ec"
server_key_version: "encryptionkey@0"
I20260807 02:27:18.693112 28523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:18.714396 28523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:18.717671 28523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:18.717895 28523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:18.718523 28523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:18.719408 28523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:18.719477 28523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.719558 28523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:18.719604 28523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:18.729658 28523 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38235
I20260807 02:27:18.729696 28708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38235 every 8 connection(s)
I20260807 02:27:18.730747 28523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:18.735590 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 28523
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.60@KRBTEST.COM
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.61@KRBTEST.COM
Aug 07 02:27:18 dist-test-slave-g170 krb5kdc[26726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069638, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:18.750834 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.732788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:42635 (local address 127.17.53.60:45675)
0807 02:27:18.732951 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.732955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.734888 (+ 1933us) server_negotiation.cc:407] Connection header received
0807 02:27:18.735124 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.735127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.735194 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.735291 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.737163 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.737919 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.739890 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740243 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740320 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.745975 (+ 5655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.745979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.745982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.746007 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.748035 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.748729 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.748736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.748740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.748860 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749342 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.749345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.749347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.749585 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.749684 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.750494 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.750638 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:27:18.752532 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.732801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:54241 (local address 127.17.53.62:40831)
0807 02:27:18.733035 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.733039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.734151 (+ 1112us) server_negotiation.cc:407] Connection header received
0807 02:27:18.735451 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.735457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.735520 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.735593 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.737190 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.737984 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.739902 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740249 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740319 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.746648 (+ 6329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.746653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.746658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.746692 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.749075 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749625 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.749631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.749636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.749725 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.750059 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.750062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.750064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.750311 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.750396 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.752239 (+ 1843us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.752352 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260807 02:27:18.753209 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.733407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:54241)
0807 02:27:18.733963 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.734020 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.734927 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.735716 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.735719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.736157 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.736983 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.736998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.738181 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.738185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.739761 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.739772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740521 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.740917 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.742727 (+ 1810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.742748 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.745075 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.749213 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749227 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.749231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.749503 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.749847 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.749852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.749949 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.750456 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.750462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.750589 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.752686 (+ 2097us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.752861 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":457,"spinlock_wait_cycles":12800,"thread_start_us":121,"threads_started":1}
I20260807 02:27:18.753760 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.733407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.17.53.3:42635)
0807 02:27:18.734795 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.734806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.734931 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.735549 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.735560 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.736160 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.736975 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.736997 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.738111 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.738114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.739741 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.739753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740522 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.740930 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.741864 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.741878 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.745827 (+ 3949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.748161 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.748167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.748185 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.748585 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.748997 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.749008 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.749215 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.749780 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.749788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.750168 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.751286 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.751711 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1279,"mutex_wait_us":308,"spinlock_wait_cycles":31360,"thread_start_us":134,"threads_started":1}
I20260807 02:27:18.754222 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.734482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:38205 (local address 127.17.53.61:38953)
0807 02:27:18.734645 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.734650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.735363 (+ 713us) server_negotiation.cc:407] Connection header received
0807 02:27:18.735545 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.735551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.735605 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.735654 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.737155 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.737985 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.739901 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740240 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740320 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.745605 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.745612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.745618 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.745655 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.749285 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.750314 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.750317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.750319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.750434 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.750754 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.750758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.750762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.751148 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.751246 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.753855 (+ 2609us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.753998 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:27:18.754824 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.734365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.17.53.3:38205)
0807 02:27:18.735287 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.735299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.735427 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.735850 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.735854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.736161 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:18.736982 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.736998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.738181 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.738185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.739761 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.739772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.740521 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.740525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.740919 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.741802 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:18.741828 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:18.745445 (+ 3617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:18.749949 (+ 4504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.749953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.749954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.750194 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.750544 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:18.750547 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:18.750549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:18.750630 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:18.751340 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:18.751345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:18.751471 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.754435 (+ 2964us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.754594 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":748,"spinlock_wait_cycles":16768,"thread_start_us":78,"threads_started":1}
I20260807 02:27:18.756047 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.738630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.17.53.62:40831)
0807 02:27:18.743340 (+ 4710us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.743356 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.743370 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:18.743403 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.743406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.743460 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.743546 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.744442 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.745625 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.747396 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.747794 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.747966 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.750672 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.750675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.750678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.750728 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.753456 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.754212 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.754215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.754217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.754343 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.754664 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.754667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.754669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.754862 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.755020 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.755698 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.755871 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4605,"thread_start_us":102,"threads_started":1}
I20260807 02:27:18.756922 28712 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40831
I20260807 02:27:18.756975 28713 heartbeater.cc:347] Connected to a master server at 127.17.53.61:38953
I20260807 02:27:18.757045 28709 heartbeater.cc:347] Connected to a master server at 127.17.53.60:45675
I20260807 02:27:18.757200 28712 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.757197 28709 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.757360 28713 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:18.757879 28712 heartbeater.cc:511] Master 127.17.53.62:40831 requested a full tablet report, sending...
I20260807 02:27:18.757879 28713 heartbeater.cc:511] Master 127.17.53.61:38953 requested a full tablet report, sending...
I20260807 02:27:18.759286 27724 ts_manager.cc:194] Registered new tserver with Master: 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:18.759439 26797 ts_manager.cc:194] Re-registered known tserver with Master: 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:18.760246 26797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:54241
I20260807 02:27:18.763089 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.737776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.17.53.60:45675)
0807 02:27:18.739653 (+ 1877us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.739658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.740017 (+ 359us) server_negotiation.cc:407] Connection header received
0807 02:27:18.740049 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.740052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.740093 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.740166 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.741427 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.742048 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.743380 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.743731 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.743843 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.758859 (+ 15016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.758866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.758873 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.758907 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.760827 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.761396 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.761403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.761407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.761511 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.761865 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.761869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.761872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.762129 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.762261 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.762682 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.762841 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1801,"thread_start_us":62,"threads_started":1}
I20260807 02:27:18.766239 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.739111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.17.53.61:38953)
0807 02:27:18.740354 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.740359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.744598 (+ 4239us) server_negotiation.cc:407] Connection header received
0807 02:27:18.744647 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.744650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.744701 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.744911 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:18.751623 (+ 6712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.752409 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.755482 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.756554 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.756653 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.759902 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:18.759908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:18.759912 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:18.759939 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:18.764370 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.764889 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.764892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.764894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.764987 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:18.765284 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:18.765288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:18.765289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:18.765474 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:18.765618 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.765901 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.766031 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":66,"threads_started":1}
I20260807 02:27:18.763849 26797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 02:27:18.769490 28709 heartbeater.cc:511] Master 127.17.53.60:45675 requested a full tablet report, sending...
W20260807 02:27:18.769731 26797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:18.770368 27994 ts_manager.cc:194] Registered new tserver with Master: 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:18.772578 27942 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: NON_VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: true } } }
I20260807 02:27:18.772727 27942 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.776217 27942 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: attempting to promote NON_VOTER 7e38c210c8d84e18b599671cd60c0ef4 to VOTER
I20260807 02:27:18.776549 27942 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:18.777705 28010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 LEARNER]: Refusing update from remote peer dd0776d83279461e9688ce1ea8d11ae1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 02:27:18.777949 27736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Refusing update from remote peer dd0776d83279461e9688ce1ea8d11ae1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 02:27:18.778303 27942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 02:27:18.778599 27942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 02:27:18.780565 28743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.781266 27736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.781396 28010 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.782262 28743 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.782378 28743 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.782518 28743 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.782644 28743 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.782824 28743 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.782912 28743 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:18.783671 28536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.783790 28536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:18.785374 28764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd0776d83279461e9688ce1ea8d11ae1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:18.785480 28764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:18.792693 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.786731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36661 (local address 127.0.0.1:42724)
0807 02:27:18.786915 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.786933 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.787027 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.788435 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.788438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.788456 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:18.788566 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.788574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.789993 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.789996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.791241 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.791250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.792262 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.792266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.792358 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.792386 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.792450 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.792525 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":62}
I20260807 02:27:18.793002 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.786940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38235 (local address 127.0.0.1:33782)
0807 02:27:18.787205 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.787219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.787305 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.788748 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.788751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.788816 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:18.788911 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.788918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.790119 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.790126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.791025 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.791036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.792463 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.792466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.792544 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.792563 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.792638 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.792717 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":24,"thread_start_us":115,"threads_started":1}
I20260807 02:27:18.793285 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.787232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.17.53.3:38235)
0807 02:27:18.788260 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.788272 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.788295 (+ 23us) server_negotiation.cc:407] Connection header received
0807 02:27:18.788380 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.788388 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.788596 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.788768 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:18.789016 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.789932 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.791166 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.792068 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.792193 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.792389 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.792720 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.792994 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":96,"threads_started":1}
I20260807 02:27:18.793640 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.787129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.17.53.1:36661)
0807 02:27:18.787560 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.787583 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.787604 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:18.787729 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.787735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.788290 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.788469 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:18.788679 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.789857 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.791371 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.792112 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.793024 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.793330 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.793392 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.793487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":91,"threads_started":1}
I20260807 02:27:18.793833 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.787275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:34243 (local address 127.0.0.1:39534)
0807 02:27:18.787891 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:18.787904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:18.787990 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:18.789464 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:18.789467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:18.789487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:18.789575 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.789581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.790970 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.790973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.792046 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:18.792055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.793263 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.793266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:18.793438 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.793462 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:27:18.793540 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 02:27:18.793605 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":101,"threads_started":1}
I20260807 02:27:18.794035 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:18.787714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39534 (local address 127.17.53.2:34243)
0807 02:27:18.789009 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:18.789020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:18.789044 (+ 24us) server_negotiation.cc:407] Connection header received
0807 02:27:18.789124 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:18.789129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:18.789314 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:18.789465 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:18.789694 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.790818 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.792213 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:18.793106 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:18.793264 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:18.793420 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:18.793574 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 02:27:18.793682 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184,"thread_start_us":101,"threads_started":1}
I20260807 02:27:18.797262 28404 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 (DEFAULT_TABLE table=test-table [id=7785e2b2714c4810af9168c27f7d0d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:18.797255 28177 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 (DEFAULT_TABLE table=test-table [id=7785e2b2714c4810af9168c27f7d0d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:18.798338 28404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5688c1faf84e87901aefdb9725a9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:18.798375 28177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5688c1faf84e87901aefdb9725a9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:18.803253 28616 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 (DEFAULT_TABLE table=test-table [id=7785e2b2714c4810af9168c27f7d0d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:18.804481 28616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5688c1faf84e87901aefdb9725a9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:18.806041 28778 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Bootstrap starting.
I20260807 02:27:18.806710 28780 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Bootstrap starting.
I20260807 02:27:18.809037 28778 tablet_bootstrap.cc:654] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:18.809037 28780 tablet_bootstrap.cc:654] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:18.809957 28778 log.cc:824] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:18.809957 28780 log.cc:824] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:18.812094 28778 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: No bootstrap required, opened a new log
I20260807 02:27:18.812094 28780 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: No bootstrap required, opened a new log
I20260807 02:27:18.812304 28778 ts_tablet_manager.cc:1397] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:27:18.812304 28780 ts_tablet_manager.cc:1397] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:27:18.812750 28786 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Bootstrap starting.
I20260807 02:27:18.815213 28786 tablet_bootstrap.cc:654] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:18.816020 28786 log.cc:824] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:18.816987 28778 raft_consensus.cc:348] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.816988 28780 raft_consensus.cc:348] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.817270 28778 raft_consensus.cc:374] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:18.817270 28780 raft_consensus.cc:374] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:18.817332 28778 raft_consensus.cc:729] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e05f0338254057b9cc200b6748afa3, State: Initialized, Role: FOLLOWER
I20260807 02:27:18.817342 28780 raft_consensus.cc:729] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23313e90d16e45e4a960fbe45ef3a2e6, State: Initialized, Role: FOLLOWER
I20260807 02:27:18.817829 28780 consensus_queue.cc:260] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.817829 28778 consensus_queue.cc:260] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.818539 28280 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
I20260807 02:27:18.818910 28778 ts_tablet_manager.cc:1428] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 02:27:18.818913 28780 ts_tablet_manager.cc:1428] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 02:27:18.819693 28511 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
I20260807 02:27:18.822208 28786 tablet_bootstrap.cc:492] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: No bootstrap required, opened a new log
I20260807 02:27:18.822522 28786 ts_tablet_manager.cc:1397] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260807 02:27:18.828087 28786 raft_consensus.cc:348] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.828377 28786 raft_consensus.cc:374] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:18.828480 28786 raft_consensus.cc:729] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9463ecf8e1df4087a0f0cc9db9055659, State: Initialized, Role: FOLLOWER
I20260807 02:27:18.829083 28786 consensus_queue.cc:260] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:18.829898 28712 heartbeater.cc:503] Master 127.17.53.62:40831 was elected leader, sending a full tablet report...
I20260807 02:27:18.830292 28786 ts_tablet_manager.cc:1428] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
W20260807 02:27:18.982335 28714 tablet.cc:2369] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:19.004419 28514 tablet.cc:2369] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:19.029592 28283 tablet.cc:2369] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:19.123034 28789 raft_consensus.cc:482] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:19.123322 28789 raft_consensus.cc:504] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:19.124894 28789 leader_election.cc:288] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661), 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:19.130128 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.125305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:49741 (local address 127.17.53.1:36661)
0807 02:27:19.125476 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.125482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.126090 (+ 608us) server_negotiation.cc:407] Connection header received
0807 02:27:19.126325 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.126328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.126383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.126466 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:19.126783 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.127349 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.128925 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.129535 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.129641 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.129717 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.129921 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.129998 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 02:27:19.130216 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.125887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:53259 (local address 127.17.53.3:38235)
0807 02:27:19.126040 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.126047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.126192 (+ 145us) server_negotiation.cc:407] Connection header received
0807 02:27:19.126373 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.126377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.126436 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.126506 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:19.126900 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.127689 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.129134 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.129548 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.129671 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.129734 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.130015 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.130092 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:27:19.130169 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.125220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36661 (local address 127.17.53.2:49741)
0807 02:27:19.126001 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:19.126016 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:19.126213 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:19.126492 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:19.126494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:19.126510 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:19.126640 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:19.126645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.127476 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.127481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:19.128797 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:19.128808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.129676 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.129679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:19.129783 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.129809 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:27:19.129888 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 02:27:19.129993 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":23,"thread_start_us":103,"threads_started":1}
I20260807 02:27:19.130640 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.125689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38235 (local address 127.17.53.2:53259)
0807 02:27:19.126081 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:19.126096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:19.126231 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:19.126616 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:19.126619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:19.126644 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:19.126773 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:19.126779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.127812 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.127816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:19.129006 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:19.129014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.129788 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.129791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:19.129886 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.129907 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:27:19.130370 (+ 463us) client_negotiation.cc:240] Negotiation successful
0807 02:27:19.130468 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":64,"spinlock_wait_cycles":896,"thread_start_us":84,"threads_started":1}
I20260807 02:27:19.131317 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d5688c1faf84e87901aefdb9725a9e5" candidate_uuid: "51e05f0338254057b9cc200b6748afa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" is_pre_election: true
I20260807 02:27:19.131426 28643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d5688c1faf84e87901aefdb9725a9e5" candidate_uuid: "51e05f0338254057b9cc200b6748afa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9463ecf8e1df4087a0f0cc9db9055659" is_pre_election: true
I20260807 02:27:19.131784 28643 raft_consensus.cc:2454] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51e05f0338254057b9cc200b6748afa3 in term 0.
I20260807 02:27:19.131803 28209 raft_consensus.cc:2454] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51e05f0338254057b9cc200b6748afa3 in term 0.
I20260807 02:27:19.132318 28315 leader_election.cc:302] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51e05f0338254057b9cc200b6748afa3, 9463ecf8e1df4087a0f0cc9db9055659; no voters:
I20260807 02:27:19.132687 28789 raft_consensus.cc:2781] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:19.132812 28789 raft_consensus.cc:482] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:19.132874 28789 raft_consensus.cc:3037] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:19.133944 28789 raft_consensus.cc:504] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:19.134303 28789 leader_election.cc:288] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [CANDIDATE]: Term 1 election: Requested vote from peers 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661), 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235)
I20260807 02:27:19.134665 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d5688c1faf84e87901aefdb9725a9e5" candidate_uuid: "51e05f0338254057b9cc200b6748afa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
I20260807 02:27:19.134721 28643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d5688c1faf84e87901aefdb9725a9e5" candidate_uuid: "51e05f0338254057b9cc200b6748afa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9463ecf8e1df4087a0f0cc9db9055659"
I20260807 02:27:19.134816 28209 raft_consensus.cc:3037] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:19.134826 28643 raft_consensus.cc:3037] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:19.136345 28209 raft_consensus.cc:2454] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51e05f0338254057b9cc200b6748afa3 in term 1.
I20260807 02:27:19.136337 28643 raft_consensus.cc:2454] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51e05f0338254057b9cc200b6748afa3 in term 1.
I20260807 02:27:19.136626 28315 leader_election.cc:302] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23313e90d16e45e4a960fbe45ef3a2e6, 51e05f0338254057b9cc200b6748afa3; no voters:
I20260807 02:27:19.136861 28789 raft_consensus.cc:2781] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:19.137063 28789 raft_consensus.cc:686] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 1 LEADER]: Becoming Leader. State: Replica: 51e05f0338254057b9cc200b6748afa3, State: Running, Role: LEADER
I20260807 02:27:19.137461 28789 consensus_queue.cc:237] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:19.140667 26795 catalog_manager.cc:5495] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51e05f0338254057b9cc200b6748afa3 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "51e05f0338254057b9cc200b6748afa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } health_report { overall_health: HEALTHY } } }
I20260807 02:27:19.155576 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.151995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.17.53.2:34243)
0807 02:27:19.152147 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.152151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.152165 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:19.152335 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.152337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.152394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.152479 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:19.152701 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.153271 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.154192 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.154423 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.154506 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.154786 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:19.155186 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:19.155278 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.155353 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.155403 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:27:19.161593 28643 raft_consensus.cc:1261] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Refusing update from remote peer 51e05f0338254057b9cc200b6748afa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:19.161593 28209 raft_consensus.cc:1261] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Refusing update from remote peer 51e05f0338254057b9cc200b6748afa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:19.162426 28795 consensus_queue.cc:1035] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:19.162664 28789 consensus_queue.cc:1035] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:19.171707 28799 mvcc.cc:205] Tried to move back new op lower bound from 7315741241997967360 to 7315741241909964800. Current Snapshot: MvccSnapshot[applied={T|T < 7315741241997967360}]
I20260807 02:27:19.174752 28800 mvcc.cc:205] Tried to move back new op lower bound from 7315741241997967360 to 7315741241909964800. Current Snapshot: MvccSnapshot[applied={T|T < 7315741241997967360}]
I20260807 02:27:19.185010 26795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:19.185199 26795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:19.190785 26795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd0776d83279461e9688ce1ea8d11ae1: Sending DeleteTablet for 3 replicas of tablet 4d5688c1faf84e87901aefdb9725a9e5
I20260807 02:27:19.191720 28177 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:19 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:42724
I20260807 02:27:19.191812 28616 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:19 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:33782
I20260807 02:27:19.192025 28404 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5688c1faf84e87901aefdb9725a9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:19 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:39534
I20260807 02:27:19.192533 28811 tablet_replica.cc:330] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: stopping tablet replica
I20260807 02:27:19.192780 28813 tablet_replica.cc:330] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: stopping tablet replica
I20260807 02:27:19.192965 28811 raft_consensus.cc:2229] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:19.193298 28813 raft_consensus.cc:2229] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:19.193382 28811 raft_consensus.cc:2258] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:19.193845 28813 raft_consensus.cc:2258] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:19.194458 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 26746
I20260807 02:27:19.201560 28813 ts_tablet_manager.cc:1905] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:19.205027 28812 tablet_replica.cc:330] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: stopping tablet replica
I20260807 02:27:19.205480 28812 raft_consensus.cc:2229] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:19.205899 28812 raft_consensus.cc:2258] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:19.207762 28812 ts_tablet_manager.cc:1905] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:19.209975 28813 ts_tablet_manager.cc:1918] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:19.210104 28813 log.cc:1196] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/4d5688c1faf84e87901aefdb9725a9e5
I20260807 02:27:19.210485 28813 ts_tablet_manager.cc:1939] T 4d5688c1faf84e87901aefdb9725a9e5 P 51e05f0338254057b9cc200b6748afa3: Deleting consensus metadata
W20260807 02:27:19.213095 28316 connection.cc:535] server connection from 127.0.0.1:39534 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 02:27:19.213136 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.193007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.17.53.61:38953)
0807 02:27:19.193157 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.193162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.202599 (+ 9437us) server_negotiation.cc:407] Connection header received
0807 02:27:19.202832 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.202835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.202892 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.202950 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:19.204070 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.204983 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.206243 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.206604 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.206683 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.209292 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:19.209297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:19.209300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:19.209330 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:19.211207 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.211714 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.211724 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.211725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.211810 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.212113 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.212116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.212118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.212349 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:19.212455 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.212835 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.212967 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:27:19.214311 28812 ts_tablet_manager.cc:1918] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:19.214439 28812 log.cc:1196] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/4d5688c1faf84e87901aefdb9725a9e5
I20260807 02:27:19.214819 28812 ts_tablet_manager.cc:1939] T 4d5688c1faf84e87901aefdb9725a9e5 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting consensus metadata
W20260807 02:27:19.216265 28547 connection.cc:441] server connection from 127.0.0.1:33782 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33782 (request call id 1) could send its response
I20260807 02:27:19.217568 28811 ts_tablet_manager.cc:1905] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:19.222014 28811 ts_tablet_manager.cc:1918] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:19.222142 28811 log.cc:1196] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/4d5688c1faf84e87901aefdb9725a9e5
I20260807 02:27:19.222536 28811 ts_tablet_manager.cc:1939] T 4d5688c1faf84e87901aefdb9725a9e5 P 23313e90d16e45e4a960fbe45ef3a2e6: Deleting consensus metadata
W20260807 02:27:19.223866 28099 connection.cc:441] server connection from 127.0.0.1:42724 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42724 (request call id 1) could send its response
I20260807 02:27:19.224113 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.194065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.17.53.60:45675)
0807 02:27:19.194177 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.194182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.213864 (+ 19682us) server_negotiation.cc:407] Connection header received
0807 02:27:19.214059 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.214062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.214117 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.214168 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:19.215342 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.216160 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.217395 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.217765 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.217889 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.219639 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:19.219643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:19.219647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:19.219681 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:19.221696 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.222569 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.222576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.222579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.222690 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.223085 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.223089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.223090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.223308 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:19.223421 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.223788 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.223953 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 02:27:19.232450 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.221171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.17.53.60:45675)
0807 02:27:19.221294 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.221299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.222265 (+ 966us) server_negotiation.cc:407] Connection header received
0807 02:27:19.222436 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.222439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.222486 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.222534 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:19.223695 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.224483 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.225703 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.226048 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.226121 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.228885 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:19.228889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:19.228892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:19.228925 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:19.230559 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.231083 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.231087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.231089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.231176 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.231472 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.231476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.231477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.231721 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:19.231879 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.232128 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.232250 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:19.234256 28738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:19.221697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.17.53.61:38953)
0807 02:27:19.221839 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:19.221844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:19.223559 (+ 1715us) server_negotiation.cc:407] Connection header received
0807 02:27:19.224992 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:19.224995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:19.225054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:19.225110 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:19.226057 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.226627 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.227839 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:19.228066 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:19.228207 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:19.230025 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:19.230029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:19.230032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:19.230061 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:19.231867 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.232948 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.232952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.232953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.233052 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:19.233399 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:19.233402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:19.233404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:19.233588 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:19.233726 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:19.233968 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 02:27:19.234093 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:27:20.176818 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.176137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:38157)
0807 02:27:20.176498 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.176609 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":150,"threads_started":1}
W20260807 02:27:20.177703 28280 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:40831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.177973 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.177183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:56233)
0807 02:27:20.177777 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.177874 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":87,"threads_started":1}
I20260807 02:27:20.178295 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.177766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:51009)
0807 02:27:20.178077 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.178176 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":69,"threads_started":1}
W20260807 02:27:20.178462 28511 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:40831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
W20260807 02:27:20.178612 28712 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:40831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.690351 28838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd0776d83279461e9688ce1ea8d11ae1)
I20260807 02:27:20.690557 28838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:20.690903 28840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd0776d83279461e9688ce1ea8d11ae1)
I20260807 02:27:20.691115 28840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:20.691823 28838 leader_election.cc:288] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675)
I20260807 02:27:20.692085 28840 leader_election.cc:288] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61:38953)
I20260807 02:27:20.692580 28845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46348)
0807 02:27:20.692388 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.692473 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260807 02:27:20.692636 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46356)
0807 02:27:20.692483 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.692546 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":64,"threads_started":1}
W20260807 02:27:20.692919 27693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
W20260807 02:27:20.692920 27973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.693778 28845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.693536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46370)
0807 02:27:20.693656 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.693710 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 02:27:20.693897 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.693601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46386)
0807 02:27:20.693748 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.693814 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260807 02:27:20.693910 27693 leader_election.cc:334] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
W20260807 02:27:20.694077 27973 leader_election.cc:334] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.696676 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58692 (local address 127.17.53.60:45675)
0807 02:27:20.693051 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:20.693055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:20.693070 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:20.693123 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:20.693130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:20.693193 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:20.693639 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:20.693762 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.694472 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.695539 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.696090 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.696211 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:20.696276 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:20.696493 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 02:27:20.696564 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":89,"threads_started":1}
I20260807 02:27:20.696677 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:45675 (local address 127.0.0.1:58692)
0807 02:27:20.692364 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.692380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:20.692496 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:20.693337 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:20.693342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:20.693366 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:20.693491 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:20.693499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.694593 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.694598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:20.695410 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:20.695419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.696209 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.696213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:20.696362 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:20.696383 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:27:20.696474 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 02:27:20.696550 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":46,"thread_start_us":76,"threads_started":1}
I20260807 02:27:20.697396 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:38953 (local address 127.0.0.1:36720)
0807 02:27:20.693656 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.693672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:20.693805 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:20.694326 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:20.694329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:20.694346 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:20.694434 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:20.694446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.695180 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.695185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:20.696145 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:20.696154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.696989 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.696994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:20.697089 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:20.697113 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:27:20.697179 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 02:27:20.697269 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":57,"threads_started":1}
I20260807 02:27:20.697690 28851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.692690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.17.53.61:38953)
0807 02:27:20.693023 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:20.693028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:20.693737 (+ 709us) server_negotiation.cc:407] Connection header received
0807 02:27:20.694149 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:20.694155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:20.694205 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:20.694315 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:20.694545 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.695067 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.696273 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:20.696853 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:20.697401 (+ 548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:20.697465 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:20.697506 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 02:27:20.697579 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":271,"thread_start_us":82,"threads_started":1}
I20260807 02:27:20.697332 28010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e38c210c8d84e18b599671cd60c0ef4" is_pre_election: true
I20260807 02:27:20.698156 28010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c5231c4d71404a9d28a8fdbae0fc55 in term 1.
I20260807 02:27:20.698423 27736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e38c210c8d84e18b599671cd60c0ef4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" is_pre_election: true
I20260807 02:27:20.698621 27736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e38c210c8d84e18b599671cd60c0ef4 in term 1.
I20260807 02:27:20.698614 27696 leader_election.cc:302] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c5231c4d71404a9d28a8fdbae0fc55, 7e38c210c8d84e18b599671cd60c0ef4; no voters: dd0776d83279461e9688ce1ea8d11ae1
I20260807 02:27:20.698907 28838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 02:27:20.698982 28838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd0776d83279461e9688ce1ea8d11ae1)
I20260807 02:27:20.699039 28838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:27:20.699218 27973 leader_election.cc:302] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c5231c4d71404a9d28a8fdbae0fc55, 7e38c210c8d84e18b599671cd60c0ef4; no voters: dd0776d83279461e9688ce1ea8d11ae1
I20260807 02:27:20.699452 28840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 02:27:20.699558 28840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd0776d83279461e9688ce1ea8d11ae1)
I20260807 02:27:20.699633 28840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:27:20.700358 28838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:20.700724 28838 leader_election.cc:288] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 election: Requested vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675)
I20260807 02:27:20.700846 28840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:20.701104 28010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e38c210c8d84e18b599671cd60c0ef4"
I20260807 02:27:20.701157 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.700900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46394)
0807 02:27:20.701049 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.701099 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 02:27:20.701232 28840 leader_election.cc:288] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 election: Requested vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 68c5231c4d71404a9d28a8fdbae0fc55 (127.17.53.61:38953)
I20260807 02:27:20.701298 28010 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 68c5231c4d71404a9d28a8fdbae0fc55 in current term 2: Already voted for candidate 7e38c210c8d84e18b599671cd60c0ef4 in this term.
I20260807 02:27:20.701633 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.701419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46400)
0807 02:27:20.701500 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.701571 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 02:27:20.701766 27736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e38c210c8d84e18b599671cd60c0ef4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68c5231c4d71404a9d28a8fdbae0fc55"
I20260807 02:27:20.701920 27736 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e38c210c8d84e18b599671cd60c0ef4 in current term 2: Already voted for candidate 68c5231c4d71404a9d28a8fdbae0fc55 in this term.
I20260807 02:27:20.702291 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.702071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46414)
0807 02:27:20.702195 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.702244 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 02:27:20.702387 27693 leader_election.cc:334] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.702443 27693 leader_election.cc:302] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68c5231c4d71404a9d28a8fdbae0fc55; no voters: 7e38c210c8d84e18b599671cd60c0ef4, dd0776d83279461e9688ce1ea8d11ae1
I20260807 02:27:20.702466 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:20.702104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46430)
0807 02:27:20.702338 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:20.702406 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260807 02:27:20.702538 27973 leader_election.cc:334] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:20.702570 28838 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 02:27:20.702584 27973 leader_election.cc:302] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e38c210c8d84e18b599671cd60c0ef4; no voters: 68c5231c4d71404a9d28a8fdbae0fc55, dd0776d83279461e9688ce1ea8d11ae1
I20260807 02:27:20.702807 28840 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 02:27:21.178932 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:21.178330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:38927)
0807 02:27:21.178689 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:21.178803 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
I20260807 02:27:21.179834 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:21.179001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:49461)
0807 02:27:21.179650 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:21.179730 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":67,"threads_started":1}
I20260807 02:27:21.179853 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:21.179384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:39123)
0807 02:27:21.179676 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:21.179742 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":62,"threads_started":1}
I20260807 02:27:22.180085 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.179667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:38845)
0807 02:27:22.179930 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.179990 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260807 02:27:22.181010 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.180501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:51683)
0807 02:27:22.180843 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.180905 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":77,"threads_started":1}
I20260807 02:27:22.181399 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.180503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:60267)
0807 02:27:22.181245 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.181309 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":69,"threads_started":1}
I20260807 02:27:22.291460 28862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:22.291659 28862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:22.292196 28862 leader_election.cc:288] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675)
I20260807 02:27:22.292719 28010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e38c210c8d84e18b599671cd60c0ef4" is_pre_election: true
I20260807 02:27:22.292929 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.292456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46594)
0807 02:27:22.292697 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.292811 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260807 02:27:22.292948 28010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68c5231c4d71404a9d28a8fdbae0fc55 in term 2.
I20260807 02:27:22.293318 27696 leader_election.cc:302] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c5231c4d71404a9d28a8fdbae0fc55, 7e38c210c8d84e18b599671cd60c0ef4; no voters:
I20260807 02:27:22.293515 28862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 02:27:22.293577 28862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:22.293605 28862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 2 FOLLOWER]: Advancing to term 3
I20260807 02:27:22.294600 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.294324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46606)
0807 02:27:22.294459 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.294518 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 02:27:22.294730 27693 leader_election.cc:334] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:22.294797 28862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:22.295138 28862 leader_election.cc:288] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 election: Requested vote from peers dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831), 7e38c210c8d84e18b599671cd60c0ef4 (127.17.53.60:45675)
I20260807 02:27:22.295532 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.295259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46620)
0807 02:27:22.295413 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.295466 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 02:27:22.295549 28010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e38c210c8d84e18b599671cd60c0ef4"
I20260807 02:27:22.295682 28010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 02:27:22.297125 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.296877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46642)
0807 02:27:22.296999 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.297056 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 02:27:22.297103 28010 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68c5231c4d71404a9d28a8fdbae0fc55 in term 3.
W20260807 02:27:22.297269 27693 leader_election.cc:334] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
I20260807 02:27:22.297403 27696 leader_election.cc:302] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 68c5231c4d71404a9d28a8fdbae0fc55, 7e38c210c8d84e18b599671cd60c0ef4; no voters: dd0776d83279461e9688ce1ea8d11ae1
I20260807 02:27:22.297554 28862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 02:27:22.297770 28862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [term 3 LEADER]: Becoming Leader. State: Replica: 68c5231c4d71404a9d28a8fdbae0fc55, State: Running, Role: LEADER
I20260807 02:27:22.298094 28862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } }
I20260807 02:27:22.299978 28865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c5231c4d71404a9d28a8fdbae0fc55. Latest consensus state: current_term: 3 leader_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:22.300101 28865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:22.300393 28868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:22.302749 28868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:22.303201 28868 catalog_manager.cc:1259] Loaded cluster ID: 2b996ed1245646dd86900b0d1d784300
I20260807 02:27:22.303260 28868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:22.303802 28868 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:22.304152 28868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Loaded TSK: 0
I20260807 02:27:22.304858 28868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:22.384326 27724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:22.387645 27724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:22.391057 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.390781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46656)
0807 02:27:22.390908 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.390976 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 02:27:22.391196 28010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [term 3 FOLLOWER]: Refusing update from remote peer 68c5231c4d71404a9d28a8fdbae0fc55: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 02:27:22.391681 28865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 02:27:22.391853 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.391565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.0.0.1:46660)
0807 02:27:22.391697 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.391754 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 02:27:22.392026 27693 consensus_peers.cc:487] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 -> Peer dd0776d83279461e9688ce1ea8d11ae1 (127.17.53.62:40831): Couldn't send request to peer dd0776d83279461e9688ce1ea8d11ae1. Status: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 02:27:22.393260 28871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c5231c4d71404a9d28a8fdbae0fc55. Latest consensus state: current_term: 3 leader_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:22.393376 28871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:22.394497 28865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:22.394642 28865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:22.394857 28871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dd0776d83279461e9688ce1ea8d11ae1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40831 } } peers { permanent_uuid: "68c5231c4d71404a9d28a8fdbae0fc55" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 38953 } attrs { promote: false } } peers { permanent_uuid: "7e38c210c8d84e18b599671cd60c0ef4" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 45675 } attrs { promote: false } } }
I20260807 02:27:22.395028 28871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e38c210c8d84e18b599671cd60c0ef4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:27:22.407065 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.401790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36661 (local address 127.0.0.1:42730)
0807 02:27:22.402039 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.402055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.402149 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.402980 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:22.402984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:22.403004 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:22.403119 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.403129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.404266 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.404270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.405560 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.405571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.406619 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.406716 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.406749 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 02:27:22.406833 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 02:27:22.406905 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":48}
I20260807 02:27:22.407297 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.402094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38235 (local address 127.0.0.1:33786)
0807 02:27:22.402867 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.402881 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.402971 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.403547 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:22.403552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:22.403578 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:22.403695 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.403704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.404712 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.404716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.405899 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.405909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.406834 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.406934 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.406965 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 02:27:22.407054 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 02:27:22.407124 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260807 02:27:22.407444 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.401959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42730 (local address 127.17.53.1:36661)
0807 02:27:22.402481 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.402487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.402506 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:27:22.402572 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.402578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.402657 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.402757 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:22.403368 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.404133 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.405706 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.406375 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406489 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.406554 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.407240 (+ 686us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.407318 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":109,"threads_started":1}
I20260807 02:27:22.407713 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.403002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33786 (local address 127.17.53.3:38235)
0807 02:27:22.403301 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.403305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.403321 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:27:22.403370 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.403373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.403420 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.403510 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:22.403830 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.404583 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406062 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.406710 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406828 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.406885 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.407088 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.407487 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":109,"threads_started":1}
I20260807 02:27:22.407981 28879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.402512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:34243 (local address 127.0.0.1:39546)
0807 02:27:22.403819 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.403833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.403934 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.404205 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:22.404208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:22.404224 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:22.404341 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.404351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.405328 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.405331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.406344 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.406356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.407538 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.407545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.407668 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.407700 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 02:27:22.407778 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 02:27:22.407852 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":27,"thread_start_us":110,"threads_started":1}
I20260807 02:27:22.408090 28177 tablet_service.cc:1467] Processing CreateTablet for tablet 211e5c92a96b47a8a009808b3c753720 (DEFAULT_TABLE table=test-table [id=6792d9b057114a5da81ce0ffcaef373c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:22.408202 28616 tablet_service.cc:1467] Processing CreateTablet for tablet 211e5c92a96b47a8a009808b3c753720 (DEFAULT_TABLE table=test-table [id=6792d9b057114a5da81ce0ffcaef373c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:22.408438 28177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211e5c92a96b47a8a009808b3c753720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:22.408522 28616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211e5c92a96b47a8a009808b3c753720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:22.408579 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.403384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.17.53.2:34243)
0807 02:27:22.403669 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.403675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.403914 (+ 239us) server_negotiation.cc:407] Connection header received
0807 02:27:22.404030 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.404033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.404096 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.404197 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:22.404467 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.405189 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.406516 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.407126 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.407368 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.407467 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.408165 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.408420 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":100,"threads_started":1}
I20260807 02:27:22.410766 28884 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Bootstrap starting.
I20260807 02:27:22.411507 28885 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: Bootstrap starting.
I20260807 02:27:22.411549 28404 tablet_service.cc:1467] Processing CreateTablet for tablet 211e5c92a96b47a8a009808b3c753720 (DEFAULT_TABLE table=test-table [id=6792d9b057114a5da81ce0ffcaef373c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:22.411847 28404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211e5c92a96b47a8a009808b3c753720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:22.411860 28884 tablet_bootstrap.cc:654] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:22.412509 28885 tablet_bootstrap.cc:654] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:22.413407 28884 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: No bootstrap required, opened a new log
I20260807 02:27:22.413494 28884 ts_tablet_manager.cc:1397] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:22.413980 28885 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: No bootstrap required, opened a new log
I20260807 02:27:22.413930 28884 raft_consensus.cc:348] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.414058 28884 raft_consensus.cc:374] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:22.414103 28884 raft_consensus.cc:729] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9463ecf8e1df4087a0f0cc9db9055659, State: Initialized, Role: FOLLOWER
I20260807 02:27:22.414103 28885 ts_tablet_manager.cc:1397] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:22.414227 28884 consensus_queue.cc:260] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.414458 28891 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Bootstrap starting.
I20260807 02:27:22.414659 28884 ts_tablet_manager.cc:1428] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:22.414680 28885 raft_consensus.cc:348] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.414793 28885 raft_consensus.cc:374] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:22.414857 28885 raft_consensus.cc:729] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23313e90d16e45e4a960fbe45ef3a2e6, State: Initialized, Role: FOLLOWER
I20260807 02:27:22.415023 28885 consensus_queue.cc:260] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.415416 28885 ts_tablet_manager.cc:1428] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 02:27:22.415444 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.415149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:56757)
0807 02:27:22.415322 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.415377 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 02:27:22.415565 28891 tablet_bootstrap.cc:654] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:22.416400 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.416141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:44377)
0807 02:27:22.416260 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.416328 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 02:27:22.418107 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.417837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:48291)
0807 02:27:22.417998 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.418045 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 02:27:22.418399 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.418109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:37973)
0807 02:27:22.418261 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.418316 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 02:27:22.418505 28282 heartbeater.cc:503] Master 127.17.53.61:38953 was elected leader, sending a full tablet report...
I20260807 02:27:22.419791 28891 tablet_bootstrap.cc:492] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: No bootstrap required, opened a new log
I20260807 02:27:22.419925 28891 ts_tablet_manager.cc:1397] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 02:27:22.419955 28713 heartbeater.cc:503] Master 127.17.53.61:38953 was elected leader, sending a full tablet report...
I20260807 02:27:22.420565 28891 raft_consensus.cc:348] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.420693 28891 raft_consensus.cc:374] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:22.420728 28891 raft_consensus.cc:729] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e05f0338254057b9cc200b6748afa3, State: Initialized, Role: FOLLOWER
I20260807 02:27:22.420923 28891 consensus_queue.cc:260] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.421344 28891 ts_tablet_manager.cc:1428] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 02:27:22.422081 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.421822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:40605)
0807 02:27:22.421944 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.422007 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 02:27:22.422847 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.422583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:48475)
0807 02:27:22.422706 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.422766 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 02:27:22.423055 28513 heartbeater.cc:503] Master 127.17.53.61:38953 was elected leader, sending a full tablet report...
I20260807 02:27:22.458688 28892 raft_consensus.cc:482] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:22.458899 28892 raft_consensus.cc:504] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.460057 28892 leader_election.cc:288] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661), 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
I20260807 02:27:22.464603 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.460357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36661 (local address 127.17.53.3:49767)
0807 02:27:22.460506 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.460531 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.460640 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.461126 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:22.461128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:22.461144 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:22.461263 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.461268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.462018 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.462022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.463269 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.463280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.464166 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.464170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.464272 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.464300 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:27:22.464393 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 02:27:22.464464 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":1}
I20260807 02:27:22.464615 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.460475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:49767 (local address 127.17.53.1:36661)
0807 02:27:22.460623 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.460628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.460645 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:22.460841 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.460844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.460909 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.460987 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:22.461417 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.461915 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.463410 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.464018 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.464144 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.464215 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.464439 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.464516 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 02:27:22.465286 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211e5c92a96b47a8a009808b3c753720" candidate_uuid: "9463ecf8e1df4087a0f0cc9db9055659" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" is_pre_election: true
I20260807 02:27:22.465415 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.460939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:44175 (local address 127.17.53.2:34243)
0807 02:27:22.461083 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.461088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.461243 (+ 155us) server_negotiation.cc:407] Connection header received
0807 02:27:22.461379 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.461382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.461436 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.461519 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:22.461792 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.462560 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.464137 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.464782 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.464900 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.464987 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.465240 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.465304 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 02:27:22.465425 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.460680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:34243 (local address 127.17.53.3:44175)
0807 02:27:22.461133 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.461146 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.461285 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.461541 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:22.461549 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:22.461566 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:22.461676 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.461681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.462733 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.462737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.463977 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:22.463990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.464941 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.464947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:22.465064 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.465092 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:27:22.465206 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 02:27:22.465276 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":66,"thread_start_us":148,"threads_started":1}
I20260807 02:27:22.465458 28209 raft_consensus.cc:2454] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9463ecf8e1df4087a0f0cc9db9055659 in term 0.
I20260807 02:27:22.465878 28545 leader_election.cc:302] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23313e90d16e45e4a960fbe45ef3a2e6, 9463ecf8e1df4087a0f0cc9db9055659; no voters:
I20260807 02:27:22.466114 28434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211e5c92a96b47a8a009808b3c753720" candidate_uuid: "9463ecf8e1df4087a0f0cc9db9055659" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51e05f0338254057b9cc200b6748afa3" is_pre_election: true
I20260807 02:27:22.466195 28892 raft_consensus.cc:2781] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:22.466265 28892 raft_consensus.cc:482] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:22.466336 28892 raft_consensus.cc:3037] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:22.466375 28434 raft_consensus.cc:2454] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9463ecf8e1df4087a0f0cc9db9055659 in term 0.
I20260807 02:27:22.467464 28892 raft_consensus.cc:504] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.467811 28892 leader_election.cc:288] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [CANDIDATE]: Term 1 election: Requested vote from peers 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661), 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243)
I20260807 02:27:22.468212 28209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211e5c92a96b47a8a009808b3c753720" candidate_uuid: "9463ecf8e1df4087a0f0cc9db9055659" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23313e90d16e45e4a960fbe45ef3a2e6"
I20260807 02:27:22.468356 28209 raft_consensus.cc:3037] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:22.468343 28434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211e5c92a96b47a8a009808b3c753720" candidate_uuid: "9463ecf8e1df4087a0f0cc9db9055659" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51e05f0338254057b9cc200b6748afa3"
I20260807 02:27:22.468467 28434 raft_consensus.cc:3037] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:22.469771 28209 raft_consensus.cc:2454] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9463ecf8e1df4087a0f0cc9db9055659 in term 1.
I20260807 02:27:22.469771 28434 raft_consensus.cc:2454] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9463ecf8e1df4087a0f0cc9db9055659 in term 1.
I20260807 02:27:22.470077 28545 leader_election.cc:302] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23313e90d16e45e4a960fbe45ef3a2e6, 9463ecf8e1df4087a0f0cc9db9055659; no voters:
I20260807 02:27:22.470314 28892 raft_consensus.cc:2781] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:22.470582 28892 raft_consensus.cc:686] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 LEADER]: Becoming Leader. State: Replica: 9463ecf8e1df4087a0f0cc9db9055659, State: Running, Role: LEADER
I20260807 02:27:22.470868 28892 consensus_queue.cc:237] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } }
I20260807 02:27:22.471280 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.471061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:45571)
0807 02:27:22.471169 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.471224 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 02:27:22.473345 27722 catalog_manager.cc:5495] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "9463ecf8e1df4087a0f0cc9db9055659" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9463ecf8e1df4087a0f0cc9db9055659" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:22.486974 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.483294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.17.53.3:38235)
0807 02:27:22.483411 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:22.483414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:22.483428 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:22.483553 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:22.483555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:22.483611 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:22.483699 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:22.483970 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.484484 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.485436 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:22.485668 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:22.485758 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:22.486093 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:22.486563 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:22.486659 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:22.486817 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 02:27:22.486869 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:27:22.490165 28209 raft_consensus.cc:1261] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6 [term 1 FOLLOWER]: Refusing update from remote peer 9463ecf8e1df4087a0f0cc9db9055659: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:22.490566 28434 raft_consensus.cc:1261] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Refusing update from remote peer 9463ecf8e1df4087a0f0cc9db9055659: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:22.490728 28892 consensus_queue.cc:1035] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23313e90d16e45e4a960fbe45ef3a2e6" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:22.491189 28900 consensus_queue.cc:1035] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51e05f0338254057b9cc200b6748afa3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:22.493261 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.492937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:48571)
0807 02:27:22.493115 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.493183 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 02:27:22.493312 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.492932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:49613)
0807 02:27:22.493123 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.493192 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 02:27:22.496265 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.495536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:38089)
0807 02:27:22.496114 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.496179 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482}
I20260807 02:27:22.497449 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.496999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.2:33927)
0807 02:27:22.497300 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.497368 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260807 02:27:22.499583 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.499223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.3:43959)
0807 02:27:22.499450 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.499510 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 02:27:22.500046 28910 mvcc.cc:205] Tried to move back new op lower bound from 7315741255634288640 to 7315741255563468800. Current Snapshot: MvccSnapshot[applied={T|T < 7315741255634288640}]
I20260807 02:27:22.501135 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:22.500740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40831 (local address 127.17.53.1:58117)
0807 02:27:22.500987 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.501041 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 02:27:22.507848 27722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:22.508030 27722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:22.513078 27722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68c5231c4d71404a9d28a8fdbae0fc55: Sending DeleteTablet for 3 replicas of tablet 211e5c92a96b47a8a009808b3c753720
I20260807 02:27:22.513773 28177 tablet_service.cc:1514] Processing DeleteTablet for tablet 211e5c92a96b47a8a009808b3c753720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:22 UTC) from {username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:42730
I20260807 02:27:22.514026 28404 tablet_service.cc:1514] Processing DeleteTablet for tablet 211e5c92a96b47a8a009808b3c753720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:22 UTC) from {username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:39546
I20260807 02:27:22.514113 28616 tablet_service.cc:1514] Processing DeleteTablet for tablet 211e5c92a96b47a8a009808b3c753720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:22 UTC) from {username='oryx', principal='oryx/127.17.53.61@KRBTEST.COM'} at 127.0.0.1:33786
I20260807 02:27:22.514448 28928 tablet_replica.cc:330] T 211e5c92a96b47a8a009808b3c753720 P 23313e90d16e45e4a960fbe45ef3a2e6: stopping tablet replica
I20260807 02:27:22.514545 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 27936
I20260807 02:27:22.525884 28929 tablet_replica.cc:330] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: stopping tablet replica
I20260807 02:27:22.526088 28929 raft_consensus.cc:2229] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:22.526271 28929 raft_consensus.cc:2258] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:22.526562 28930 tablet_replica.cc:330] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: stopping tablet replica
I20260807 02:27:22.526717 28930 raft_consensus.cc:2229] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:22.527148 28930 raft_consensus.cc:2258] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:22.527475 28929 ts_tablet_manager.cc:1905] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:22.527930 28930 ts_tablet_manager.cc:1905] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:22.530668 28929 ts_tablet_manager.cc:1918] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:22.530872 28929 log.cc:1196] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/211e5c92a96b47a8a009808b3c753720
I20260807 02:27:22.531332 28929 ts_tablet_manager.cc:1939] T 211e5c92a96b47a8a009808b3c753720 P 51e05f0338254057b9cc200b6748afa3: Deleting consensus metadata
I20260807 02:27:22.531317 28930 ts_tablet_manager.cc:1918] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:22.531407 28930 log.cc:1196] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/211e5c92a96b47a8a009808b3c753720
I20260807 02:27:22.531747 28930 ts_tablet_manager.cc:1939] T 211e5c92a96b47a8a009808b3c753720 P 9463ecf8e1df4087a0f0cc9db9055659: Deleting consensus metadata
I20260807 02:27:22.532618 27689 catalog_manager.cc:4841] TS 9463ecf8e1df4087a0f0cc9db9055659 (127.17.53.3:38235): tablet 211e5c92a96b47a8a009808b3c753720 (table test-table [id=6792d9b057114a5da81ce0ffcaef373c]) successfully deleted
I20260807 02:27:22.532908 27689 catalog_manager.cc:4841] TS 51e05f0338254057b9cc200b6748afa3 (127.17.53.2:34243): tablet 211e5c92a96b47a8a009808b3c753720 (table test-table [id=6792d9b057114a5da81ce0ffcaef373c]) successfully deleted
W20260807 02:27:22.533843 28881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:22.533185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36661 (local address 127.0.0.1:42734)
0807 02:27:22.533316 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:22.533334 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:22.533422 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:22.533704 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:36661: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 02:27:22.534029 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 28290
W20260807 02:27:22.534037 27689 catalog_manager.cc:4584] TS 23313e90d16e45e4a960fbe45ef3a2e6 (127.17.53.1:36661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 211e5c92a96b47a8a009808b3c753720: Network error: Client connection negotiation failed: client connection to 127.17.53.1:36661: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 02:27:22.547322 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 28523
I20260807 02:27:22.562608 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 27507
I20260807 02:27:22.575896 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 27669
2026-08-07T02:27:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7697 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:22 dist-test-slave-g170 krb5kdc[28934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:22 dist-test-slave-g170 krb5kdc[28934](info): set up 2 sockets
Aug 07 02:27:22 dist-test-slave-g170 krb5kdc[28934](info): commencing operation
krb5kdc: starting...
W20260807 02:27:23.851342 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:23 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab.
Entry for principal oryx/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab.
I20260807 02:27:23.990041 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40867
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab
--principal=oryx/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:40867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:24.106189 28956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:24.106504 28956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:24.106586 28956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:24.110706 28956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:24.110810 28956 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 02:27:24.110843 28956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:24.110890 28956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:24.110949 28956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:24.110991 28956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:24.116348 28956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:40867
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.62.keytab
--principal=oryx/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:40867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:24.117558 28956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:24.118686 28956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:24.128015 28963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.128113 28962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.128036 28965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:24.128338 28956 server_base.cc:1034] running on GCE node
I20260807 02:27:24.129022 28956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:24.130167 28956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:24.131357 28956 hybrid_clock.cc:648] HybridClock initialized: now 1786069644131329 us; error 42 us; skew 500 ppm
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:24.134222 28956 init.cc:377] Logged in from keytab as oryx/127.17.53.62@KRBTEST.COM (short username oryx)
I20260807 02:27:24.135142 28956 webserver.cc:460] Webserver started at http://127.17.53.62:37031/ using document root <none> and password file <none>
I20260807 02:27:24.135679 28956 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:24.135737 28956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:24.135913 28956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:24.137952 28956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "c567405852b44e96b39d2eb05dce5ca4"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "8e322403628f5399188a54ac892d0ae8"
server_key_iv: "8a931dca3d0b694099392413bb779798"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.138471 28956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "c567405852b44e96b39d2eb05dce5ca4"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "8e322403628f5399188a54ac892d0ae8"
server_key_iv: "8a931dca3d0b694099392413bb779798"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.142180 28956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:24.144556 28975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.145951 28956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:24.146055 28956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "c567405852b44e96b39d2eb05dce5ca4"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "8e322403628f5399188a54ac892d0ae8"
server_key_iv: "8a931dca3d0b694099392413bb779798"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.146139 28956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:24.161157 28956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:24.164263 28956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:24.164424 28956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:24.172643 28956 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:40867
I20260807 02:27:24.172752 29077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:40867 every 8 connection(s)
I20260807 02:27:24.173729 28956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:24.176651 29079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:24.177762 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 28956
I20260807 02:27:24.177917 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:24.178256 17620 external_mini_cluster.cc:1428] Setting key a4180e2948a579b332a07e86a30720c2
I20260807 02:27:24.183178 29079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Bootstrap starting.
I20260807 02:27:24.185657 29079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:24.186442 29079 log.cc:824] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:24.188591 29079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: No bootstrap required, opened a new log
I20260807 02:27:24.192121 29079 raft_consensus.cc:348] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } }
I20260807 02:27:24.192399 29079 raft_consensus.cc:374] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:24.192494 29079 raft_consensus.cc:729] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c567405852b44e96b39d2eb05dce5ca4, State: Initialized, Role: FOLLOWER
I20260807 02:27:24.192863 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.179775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.17.53.62:40867)
0807 02:27:24.180225 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.180241 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.180285 (+ 44us) server_negotiation.cc:407] Connection header received
0807 02:27:24.180933 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.180950 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.181288 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.181625 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.182474 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.183521 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.184372 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.184913 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.185096 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.187876 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.187890 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.187904 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.187930 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.189744 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.190204 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.190217 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.190222 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.190353 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.190651 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.190655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.190657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.191061 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.191290 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.191571 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.191867 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":140,"threads_started":1}
I20260807 02:27:24.193071 29079 consensus_queue.cc:260] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } }
I20260807 02:27:24.193248 29079 raft_consensus.cc:388] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:24.193320 29079 raft_consensus.cc:482] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:24.193423 29079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:24.194578 29079 raft_consensus.cc:504] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } }
I20260807 02:27:24.194940 29079 leader_election.cc:302] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c567405852b44e96b39d2eb05dce5ca4; no voters:
I20260807 02:27:24.195353 29079 leader_election.cc:288] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:24.195454 29086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:24.195691 29086 raft_consensus.cc:686] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [term 1 LEADER]: Becoming Leader. State: Replica: c567405852b44e96b39d2eb05dce5ca4, State: Running, Role: LEADER
I20260807 02:27:24.196036 29086 consensus_queue.cc:237] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } }
I20260807 02:27:24.196653 29079 sys_catalog.cc:564] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:24.197774 29087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c567405852b44e96b39d2eb05dce5ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } } }
I20260807 02:27:24.197768 29088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c567405852b44e96b39d2eb05dce5ca4. Latest consensus state: current_term: 1 leader_uuid: "c567405852b44e96b39d2eb05dce5ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c567405852b44e96b39d2eb05dce5ca4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 40867 } } }
I20260807 02:27:24.197918 29088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:24.197918 29087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4 [sys.catalog]: This master's current role is: LEADER
W20260807 02:27:24.201999 29108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:24.202090 29108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:24.202159 29099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:24.202864 29099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:24.208539 29099 catalog_manager.cc:1347] Generated new cluster ID: 8d976b3ce6314affa09b288b6fa6a661
I20260807 02:27:24.208611 29099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:24.221971 29099 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:24.222863 29099 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:24.229538 29099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Generated new TSK 0
I20260807 02:27:24.230250 29099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
I20260807 02:27:24.268863 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40867
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:24.393850 29116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:24.394143 29116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:24.394222 29116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:24.398439 29116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:24.398528 29116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:24.398599 29116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:27:24.398681 29116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:24.404284 29116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:24.405441 29116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:24.406591 29116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:24.417354 29127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.417354 29122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.417400 29123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:24.417761 29116 server_base.cc:1034] running on GCE node
I20260807 02:27:24.418259 29116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:24.419001 29116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:24.420210 29116 hybrid_clock.cc:648] HybridClock initialized: now 1786069644420174 us; error 47 us; skew 500 ppm
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:24.423491 29116 init.cc:377] Logged in from keytab as oryx/127.17.53.1@KRBTEST.COM (short username oryx)
I20260807 02:27:24.424914 29116 webserver.cc:460] Webserver started at http://127.17.53.1:45495/ using document root <none> and password file <none>
I20260807 02:27:24.425580 29116 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:24.425679 29116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:24.425963 29116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:24.428319 29116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "b9ddf9fae22f483a86be638644521f18"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3344ba1ea1850a4c9951b2a27488a7f1"
server_key_iv: "8bf278acd4f61d933e9fcf22bd01da85"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.429056 29116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "b9ddf9fae22f483a86be638644521f18"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3344ba1ea1850a4c9951b2a27488a7f1"
server_key_iv: "8bf278acd4f61d933e9fcf22bd01da85"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.433384 29116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:24.436189 29140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.437644 29116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:24.437793 29116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "b9ddf9fae22f483a86be638644521f18"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3344ba1ea1850a4c9951b2a27488a7f1"
server_key_iv: "8bf278acd4f61d933e9fcf22bd01da85"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.437916 29116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:24.449533 29116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:24.453122 29116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:24.453338 29116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:24.454993 29116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:24.455070 29116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.455149 29116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:24.455194 29116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:24.466519 29116 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:38799
I20260807 02:27:24.466760 29315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:38799 every 8 connection(s)
I20260807 02:27:24.467770 29116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:24.470223 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.455750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:44953 (local address 127.17.53.62:40867)
0807 02:27:24.455902 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.455907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.456865 (+ 958us) server_negotiation.cc:407] Connection header received
0807 02:27:24.457946 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.457949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.458008 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.458067 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.459992 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.460560 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.461957 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.462200 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.462329 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.465913 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.465917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.465920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.465957 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.467762 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.468406 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.468410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.468412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.468514 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.469082 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.469085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.469086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.469336 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.469427 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.469976 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.470096 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:27:24.471055 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.456183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.1:44953)
0807 02:27:24.456667 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.456730 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.457668 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.458525 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.458536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.459080 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.459824 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.459843 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.460704 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.460710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.461820 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.461832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.462387 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.462391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.462599 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.463228 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.463245 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.465724 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.467910 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.467922 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.467940 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.468261 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.468633 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.468636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.468638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.468941 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.469452 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.469457 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.469696 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.469903 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.470168 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":137,"threads_started":1}
I20260807 02:27:24.472506 29316 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40867
I20260807 02:27:24.472656 29316 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:24.473296 29316 heartbeater.cc:511] Master 127.17.53.62:40867 requested a full tablet report, sending...
I20260807 02:27:24.474402 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 29116
I20260807 02:27:24.474545 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:24.474910 17620 external_mini_cluster.cc:1428] Setting key 196e90348baf2066b37b98885ea28ddb
I20260807 02:27:24.475080 29002 ts_manager.cc:194] Registered new tserver with Master: b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799)
I20260807 02:27:24.476613 29002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:44953
I20260807 02:27:24.483484 29325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.475075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.1:57489)
0807 02:27:24.475496 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.475513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.475600 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.475949 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.475955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.476400 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.476535 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.476542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.477394 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.477402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.478254 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.478266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.478901 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.478905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.479014 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.479566 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.479578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.479915 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.481881 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.481884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.481887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.482091 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.482459 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.482461 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.482463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.482547 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.483058 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.483060 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.483138 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.483223 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.483337 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":9,"thread_start_us":117,"threads_started":1}
I20260807 02:27:24.483865 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.475423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:57489 (local address 127.17.53.62:40867)
0807 02:27:24.475764 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.475767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.475777 (+ 10us) server_negotiation.cc:407] Connection header received
0807 02:27:24.475804 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.475806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.475840 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.475930 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.476653 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.477254 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.478405 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.478765 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.478883 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.480058 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.480062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.480065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.480089 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.481695 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.482206 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.482209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.482211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.482325 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.482656 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.482663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.482665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.482946 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.483071 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.483333 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.483472 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":264}
WARNING: no policy specified for oryx/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
I20260807 02:27:24.507526 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40867
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:24.626520 29329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:24.626772 29329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:24.626817 29329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:24.630870 29329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:24.630928 29329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:24.630967 29329 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:27:24.631000 29329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:24.636243 29329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:24.637269 29329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:24.638255 29329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:24.648118 29334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.648128 29338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:24.648212 29329 server_base.cc:1034] running on GCE node
W20260807 02:27:24.648132 29335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:24.648943 29329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:24.649696 29329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:24.650883 29329 hybrid_clock.cc:648] HybridClock initialized: now 1786069644650853 us; error 39 us; skew 500 ppm
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:24.653929 29329 init.cc:377] Logged in from keytab as oryx/127.17.53.2@KRBTEST.COM (short username oryx)
I20260807 02:27:24.654847 29329 webserver.cc:460] Webserver started at http://127.17.53.2:46627/ using document root <none> and password file <none>
I20260807 02:27:24.655428 29329 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:24.655541 29329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:24.655797 29329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:24.658001 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "bbd38fedd4974e19baaecd234f0aeef1"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "620b6d913477baa2e564285d5e81d168"
server_key_iv: "99a1a83b2643bedc2fa17506420fe231"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.658598 29329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "bbd38fedd4974e19baaecd234f0aeef1"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "620b6d913477baa2e564285d5e81d168"
server_key_iv: "99a1a83b2643bedc2fa17506420fe231"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.662580 29329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:24.665191 29349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.666460 29329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:27:24.666627 29329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "bbd38fedd4974e19baaecd234f0aeef1"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "620b6d913477baa2e564285d5e81d168"
server_key_iv: "99a1a83b2643bedc2fa17506420fe231"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.666754 29329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:24.683126 29329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:24.686805 29329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:24.687036 29329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:24.688537 29329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:24.688614 29329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.688702 29329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:24.688757 29329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:24.700412 29329 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39173
I20260807 02:27:24.700484 29519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39173 every 8 connection(s)
I20260807 02:27:24.701411 29329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:24.703034 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.689476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:38125 (local address 127.17.53.62:40867)
0807 02:27:24.689630 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.689635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.690564 (+ 929us) server_negotiation.cc:407] Connection header received
0807 02:27:24.691669 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.691674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.691742 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.691835 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.693580 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.694206 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.695494 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.695808 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.695929 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.699209 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.699213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.699216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.699243 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.700924 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.701439 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.701442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.701444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.701535 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.701894 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.701897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.701899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.702150 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.702256 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.702814 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.702916 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 02:27:24.703850 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.689905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.2:38125)
0807 02:27:24.690384 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.690434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.691408 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.692289 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.692303 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.692856 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.693432 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.693447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.694347 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.694351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.695359 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.695369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.695985 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.695989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.696199 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.696948 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.696967 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.699038 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.701039 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.701045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.701060 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.701322 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.701629 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.701632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.701634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.701788 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.702255 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.702263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.702552 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.702746 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.702996 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":133,"threads_started":1}
I20260807 02:27:24.705118 29520 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40867
I20260807 02:27:24.705314 29520 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:24.705518 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 29329
I20260807 02:27:24.705657 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:24.705891 29520 heartbeater.cc:511] Master 127.17.53.62:40867 requested a full tablet report, sending...
I20260807 02:27:24.706082 17620 external_mini_cluster.cc:1428] Setting key 482147bb1e5d9088cf4e027774abfb42
I20260807 02:27:24.707162 29002 ts_manager.cc:194] Registered new tserver with Master: bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173)
I20260807 02:27:24.707918 29002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:38125
I20260807 02:27:24.715317 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.707586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.2:48959)
0807 02:27:24.707898 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.707910 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.708025 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.708390 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.708395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.708616 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.708749 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.708756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.709540 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.709544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.710379 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.710394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.710893 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.710898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.711014 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.711520 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.711539 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.711940 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.713738 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.713743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.713746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.713954 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.714245 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.714248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.714250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.714340 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.714750 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.714753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.714859 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.714970 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.715131 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
I20260807 02:27:24.715309 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.707629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:48959 (local address 127.17.53.62:40867)
0807 02:27:24.707744 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.707748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.707983 (+ 235us) server_negotiation.cc:407] Connection header received
0807 02:27:24.708141 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.708144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.708191 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.708281 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.708906 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.709409 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.710563 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.710772 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.710858 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.712088 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.712092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.712094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.712118 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.713621 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.714058 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.714062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.714064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.714150 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.714434 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.714439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.714440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.714651 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.714759 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.715027 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.715141 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
I20260807 02:27:24.736371 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:40867
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:24.857795 29533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:24.858096 29533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:24.858155 29533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:24.862435 29533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:24.862514 29533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:24.862557 29533 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 02:27:24.862632 29533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:24.868185 29533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.53.62:40867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:24.869508 29533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:24.870757 29533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:24.881831 29543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.881919 29544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:24.881827 29547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:24.882803 29533 server_base.cc:1034] running on GCE node
I20260807 02:27:24.883353 29533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:24.884115 29533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:24.885342 29533 hybrid_clock.cc:648] HybridClock initialized: now 1786069644885312 us; error 44 us; skew 500 ppm
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:24.888556 29533 init.cc:377] Logged in from keytab as oryx/127.17.53.3@KRBTEST.COM (short username oryx)
I20260807 02:27:24.889622 29533 webserver.cc:460] Webserver started at http://127.17.53.3:35475/ using document root <none> and password file <none>
I20260807 02:27:24.890221 29533 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:24.890287 29533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:24.890508 29533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:24.892724 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "bf7a525279f74dd490df667a6a1411c0"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3cdca05a64fb096b37d7e1c9a788b668"
server_key_iv: "6ec1c06cfebbc5b7a8e869d1dd20ee15"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.893388 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "bf7a525279f74dd490df667a6a1411c0"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3cdca05a64fb096b37d7e1c9a788b668"
server_key_iv: "6ec1c06cfebbc5b7a8e869d1dd20ee15"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.897763 29533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:24.900817 29557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.902221 29533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:27:24.902379 29533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "bf7a525279f74dd490df667a6a1411c0"
format_stamp: "Formatted at 2026-08-07 02:27:24 on dist-test-slave-g170"
server_key: "3cdca05a64fb096b37d7e1c9a788b668"
server_key_iv: "6ec1c06cfebbc5b7a8e869d1dd20ee15"
server_key_version: "encryptionkey@0"
I20260807 02:27:24.902513 29533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:24.924568 29533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:24.928216 29533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:24.928413 29533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:24.930016 29533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:24.930073 29533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:24.930116 29533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:24.930177 29533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:24.942618 29533 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:44605
I20260807 02:27:24.942689 29728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:44605 every 8 connection(s)
I20260807 02:27:24.943877 29533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:24.944705 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 29533
I20260807 02:27:24.944905 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:24.945302 17620 external_mini_cluster.cc:1428] Setting key 16f68a704ed123411dfdcbe38da29c42
I20260807 02:27:24.946925 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.931191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:35481 (local address 127.17.53.62:40867)
0807 02:27:24.931358 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.931362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.932697 (+ 1335us) server_negotiation.cc:407] Connection header received
0807 02:27:24.934094 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.934101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.934181 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.934311 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.936369 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.937152 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.938786 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.939181 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.939297 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.942598 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.942602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.942605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.942639 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.944488 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.945208 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.945218 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.945220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.945304 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.945690 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.945693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.945695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.945908 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.946023 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.946613 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.946740 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:27:24.947904 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.931745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.3:35481)
0807 02:27:24.932499 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.932547 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.933781 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.934598 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.934606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.935332 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.936177 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.936203 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.937384 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.937388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.938582 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.938594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.939364 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.939367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.939523 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.940232 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.940261 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.942376 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.944636 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.944644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.944665 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.945077 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.945416 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.945419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.945421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.945576 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.946037 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.946045 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.946355 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.946552 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.946891 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":151,"threads_started":1}
I20260807 02:27:24.949220 29730 heartbeater.cc:347] Connected to a master server at 127.17.53.62:40867
I20260807 02:27:24.949400 29730 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:24.950058 29730 heartbeater.cc:511] Master 127.17.53.62:40867 requested a full tablet report, sending...
I20260807 02:27:24.951184 29003 ts_manager.cc:194] Registered new tserver with Master: bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:24.951889 29003 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:35481
I20260807 02:27:24.959518 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.951732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:58367 (local address 127.17.53.62:40867)
0807 02:27:24.951865 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.951871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.952063 (+ 192us) server_negotiation.cc:407] Connection header received
0807 02:27:24.952222 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.952226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.952269 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.952362 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.952972 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.953524 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.954439 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.954691 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.954814 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.956029 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.956034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.956037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.956064 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.957501 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.957999 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.958007 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.958008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.958083 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:24.958395 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:24.958398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:24.958400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:24.958621 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:24.958784 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:24.959234 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 02:27:24.959357 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:24.959532 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.951647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.17.53.3:58367)
0807 02:27:24.951984 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:24.951995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:24.952104 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:24.952417 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:24.952419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:24.952659 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:24.952835 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.952843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.953712 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.953719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.954308 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:24.954318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.954820 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.954825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:24.954927 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.955522 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:24.955535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:24.955871 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:24.957645 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.957648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.957650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.957866 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.958179 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:24.958182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:24.958184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:24.958283 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:24.958773 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:24.958776 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:24.959007 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 02:27:24.959157 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 02:27:24.959328 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":104,"threads_started":1}
I20260807 02:27:24.959988 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 02:27:24.990351 29082 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:24.982580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.17.53.62:40867)
0807 02:27:24.982775 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.982780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.982801 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:24.982857 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.982860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.982918 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.983071 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.984055 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.984664 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.985704 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.985961 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.986101 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.990118 (+ 4017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49284: BlockingRecv error: recv got EOF from 127.0.0.1:49284 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 02:27:24 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:25.001536 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:24.992010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.17.53.62:40867)
0807 02:27:24.992137 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:24.992141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:24.992233 (+ 92us) server_negotiation.cc:407] Connection header received
0807 02:27:24.992338 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:24.992340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:24.992383 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:24.992460 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:24.993156 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.993720 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.994589 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:24.994839 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:24.994927 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:24.998053 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:24.998057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:24.998060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:24.998096 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:24.999566 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.000094 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.000097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.000099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.000194 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.000527 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.000531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.000536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.000737 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:25.000948 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.001217 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.001354 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:27:25.004907 29003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:25.008379 29003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:25.024315 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.018320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.17.53.3:44605)
0807 02:27:25.019878 (+ 1558us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.019888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.019911 (+ 23us) server_negotiation.cc:407] Connection header received
0807 02:27:25.019983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.019991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.020171 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.020320 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.020596 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.021784 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.022990 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.023558 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.023687 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.023871 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.024050 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.024147 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":139,"threads_started":1}
I20260807 02:27:25.024973 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.018142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:44605 (local address 127.0.0.1:38040)
0807 02:27:25.019357 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.019412 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.019547 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.020316 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.020319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.020337 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.020460 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.020469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.021899 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.021906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.022839 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.022849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.023687 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.023690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.023824 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.023870 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.024012 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.024142 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":29,"thread_start_us":76,"threads_started":1}
I20260807 02:27:25.026012 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.018492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.0.0.1:46346)
0807 02:27:25.019513 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.019525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.019615 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.020109 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.020115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.020141 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.020307 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.020320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.022040 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.022046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.023633 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.023641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.025474 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.025477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.025622 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.025679 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.025776 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.025861 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":106,"thread_start_us":106,"threads_started":1}
I20260807 02:27:25.026077 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.018931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.17.53.1:38799)
0807 02:27:25.019371 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.019380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.019616 (+ 236us) server_negotiation.cc:407] Connection header received
0807 02:27:25.019714 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.019721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.019947 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.020102 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.020472 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.021871 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.023802 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.024831 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.024928 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.025222 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.025823 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.025938 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":73,"threads_started":1}
I20260807 02:27:25.026396 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.019112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.17.53.2:39173)
0807 02:27:25.019511 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.019524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.019873 (+ 349us) server_negotiation.cc:407] Connection header received
0807 02:27:25.020010 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.020014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.020252 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.020427 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.020740 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.021995 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.023801 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.024718 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.025037 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.025563 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.026099 (+ 536us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.026257 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
I20260807 02:27:25.026026 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.018910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39173 (local address 127.0.0.1:60496)
0807 02:27:25.019763 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.019774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.019883 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.020483 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.020486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.020505 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.020612 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.020619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.022123 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.022127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.023629 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.023641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.025632 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.025636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.025779 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.025801 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.025864 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.025921 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":122,"thread_start_us":106,"threads_started":1}
I20260807 02:27:25.028335 29634 tablet_service.cc:1467] Processing CreateTablet for tablet 144ea806e9074749b831602c8b4fcdfe (DEFAULT_TABLE table=test-table [id=b4b371e635be4b44b0c5e057158eb877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.029147 29421 tablet_service.cc:1467] Processing CreateTablet for tablet 144ea806e9074749b831602c8b4fcdfe (DEFAULT_TABLE table=test-table [id=b4b371e635be4b44b0c5e057158eb877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.029598 29634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ea806e9074749b831602c8b4fcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.029284 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 144ea806e9074749b831602c8b4fcdfe (DEFAULT_TABLE table=test-table [id=b4b371e635be4b44b0c5e057158eb877]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.030088 29421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ea806e9074749b831602c8b4fcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.030298 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144ea806e9074749b831602c8b4fcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.036885 29777 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Bootstrap starting.
I20260807 02:27:25.038618 29779 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Bootstrap starting.
I20260807 02:27:25.039477 29777 tablet_bootstrap.cc:654] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.040359 29777 log.cc:824] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:25.040930 29779 tablet_bootstrap.cc:654] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.041122 29778 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Bootstrap starting.
I20260807 02:27:25.041802 29779 log.cc:824] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:25.042860 29777 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: No bootstrap required, opened a new log
I20260807 02:27:25.043074 29777 ts_tablet_manager.cc:1397] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 02:27:25.043223 29778 tablet_bootstrap.cc:654] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.043958 29779 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: No bootstrap required, opened a new log
I20260807 02:27:25.044013 29778 log.cc:824] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:25.044207 29779 ts_tablet_manager.cc:1397] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 02:27:25.047092 29778 tablet_bootstrap.cc:492] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: No bootstrap required, opened a new log
I20260807 02:27:25.047297 29778 ts_tablet_manager.cc:1397] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:27:25.047271 29777 raft_consensus.cc:348] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.047559 29777 raft_consensus.cc:374] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.047621 29777 raft_consensus.cc:729] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.047814 29779 raft_consensus.cc:348] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.048148 29779 raft_consensus.cc:374] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.048110 29777 consensus_queue.cc:260] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.048225 29779 raft_consensus.cc:729] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7a525279f74dd490df667a6a1411c0, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.048715 29779 consensus_queue.cc:260] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.049389 29777 ts_tablet_manager.cc:1428] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260807 02:27:25.049525 29730 heartbeater.cc:503] Master 127.17.53.62:40867 was elected leader, sending a full tablet report...
I20260807 02:27:25.049752 29520 heartbeater.cc:503] Master 127.17.53.62:40867 was elected leader, sending a full tablet report...
I20260807 02:27:25.049901 29779 ts_tablet_manager.cc:1428] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 02:27:25.052327 29778 raft_consensus.cc:348] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.052551 29778 raft_consensus.cc:374] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.052644 29778 raft_consensus.cc:729] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.053133 29778 consensus_queue.cc:260] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.054097 29778 ts_tablet_manager.cc:1428] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260807 02:27:25.054154 29316 heartbeater.cc:503] Master 127.17.53.62:40867 was elected leader, sending a full tablet report...
I20260807 02:27:25.059304 29784 raft_consensus.cc:482] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:25.059432 29784 raft_consensus.cc:504] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.060467 29784 leader_election.cc:288] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605), b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799)
I20260807 02:27:25.064141 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.060653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:44605 (local address 127.17.53.2:47209)
0807 02:27:25.060865 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.060879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.060970 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.061261 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.061264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.061283 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.061423 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.061428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.062213 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.062219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.063052 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.063061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.063698 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.063710 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.063821 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.063849 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.063937 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.064013 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":21,"spinlock_wait_cycles":23808}
I20260807 02:27:25.064136 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.060797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:47209 (local address 127.17.53.3:44605)
0807 02:27:25.061035 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.061042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.061057 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:25.061105 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.061107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.061142 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.061246 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.061532 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.062097 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.063174 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.063573 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.063695 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.063759 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.063954 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.064020 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260807 02:27:25.064800 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.060780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.17.53.2:51591)
0807 02:27:25.061189 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.061200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.061337 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.061586 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.061588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.061604 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.061746 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.061751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.062459 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.062463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.063641 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.063651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.064351 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.064355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.064467 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.064491 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.064570 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.064638 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":21,"thread_start_us":130,"threads_started":1}
I20260807 02:27:25.064850 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.060877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:51591 (local address 127.17.53.1:38799)
0807 02:27:25.061015 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.061019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.061277 (+ 258us) server_negotiation.cc:407] Connection header received
0807 02:27:25.061447 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.061450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.061487 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.061574 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.061857 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.062331 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.063777 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.064220 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.064347 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.064399 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.064616 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.064705 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:27:25.064862 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ea806e9074749b831602c8b4fcdfe" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0" is_pre_election: true
I20260807 02:27:25.065181 29667 raft_consensus.cc:2454] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 0.
I20260807 02:27:25.065806 29354 leader_election.cc:302] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd38fedd4974e19baaecd234f0aeef1, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.065945 29249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ea806e9074749b831602c8b4fcdfe" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ddf9fae22f483a86be638644521f18" is_pre_election: true
I20260807 02:27:25.066111 29784 raft_consensus.cc:2781] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:25.066200 29784 raft_consensus.cc:482] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:25.066241 29784 raft_consensus.cc:3037] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.066239 29249 raft_consensus.cc:2454] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 0.
I20260807 02:27:25.067230 29784 raft_consensus.cc:504] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.067564 29784 leader_election.cc:288] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 election: Requested vote from peers bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605), b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799)
I20260807 02:27:25.067914 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ea806e9074749b831602c8b4fcdfe" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0"
I20260807 02:27:25.068094 29249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144ea806e9074749b831602c8b4fcdfe" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ddf9fae22f483a86be638644521f18"
I20260807 02:27:25.068094 29667 raft_consensus.cc:3037] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.068233 29249 raft_consensus.cc:3037] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.069739 29667 raft_consensus.cc:2454] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 1.
I20260807 02:27:25.069815 29249 raft_consensus.cc:2454] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 1.
I20260807 02:27:25.070176 29354 leader_election.cc:302] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd38fedd4974e19baaecd234f0aeef1, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.070451 29784 raft_consensus.cc:2781] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:25.070719 29784 raft_consensus.cc:686] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 1 LEADER]: Becoming Leader. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Running, Role: LEADER
I20260807 02:27:25.071085 29784 consensus_queue.cc:237] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } }
I20260807 02:27:25.074982 29001 catalog_manager.cc:5495] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "bbd38fedd4974e19baaecd234f0aeef1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:25.086390 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.082935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.17.53.2:39173)
0807 02:27:25.083054 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.083058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.083071 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:27:25.083207 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.083209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.083262 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.083350 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.083560 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.084059 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.084960 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.085190 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.085272 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.085606 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.086021 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.086123 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.086199 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.086273 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:27:25.092687 29249 raft_consensus.cc:1261] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Refusing update from remote peer bbd38fedd4974e19baaecd234f0aeef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.092998 29667 raft_consensus.cc:1261] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Refusing update from remote peer bbd38fedd4974e19baaecd234f0aeef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.093472 29784 consensus_queue.cc:1035] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.093685 29788 consensus_queue.cc:1035] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.111056 29003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.111243 29003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.114053 29003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Sending DeleteTablet for 3 replicas of tablet 144ea806e9074749b831602c8b4fcdfe
I20260807 02:27:25.114910 29207 tablet_service.cc:1514] Processing DeleteTablet for tablet 144ea806e9074749b831602c8b4fcdfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260807 02:27:25.114948 29421 tablet_service.cc:1514] Processing DeleteTablet for tablet 144ea806e9074749b831602c8b4fcdfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60496
I20260807 02:27:25.115172 29634 tablet_service.cc:1514] Processing DeleteTablet for tablet 144ea806e9074749b831602c8b4fcdfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 02:27:25.115468 29807 tablet_replica.cc:330] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: stopping tablet replica
I20260807 02:27:25.115507 29805 tablet_replica.cc:330] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: stopping tablet replica
I20260807 02:27:25.115792 29807 raft_consensus.cc:2229] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.115826 29805 raft_consensus.cc:2229] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:25.116094 29807 raft_consensus.cc:2258] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.116084 29003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 02:27:25.116230 29805 raft_consensus.cc:2258] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 02:27:25.116523 29003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:25.116598 29812 tablet_replica.cc:330] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: stopping tablet replica
I20260807 02:27:25.116933 29812 raft_consensus.cc:2229] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.117220 29812 raft_consensus.cc:2258] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.117714 29805 ts_tablet_manager.cc:1905] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.119664 29807 ts_tablet_manager.cc:1905] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.121028 29805 ts_tablet_manager.cc:1918] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.121131 29805 log.cc:1196] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/144ea806e9074749b831602c8b4fcdfe
I20260807 02:27:25.121462 29805 ts_tablet_manager.cc:1939] T 144ea806e9074749b831602c8b4fcdfe P bbd38fedd4974e19baaecd234f0aeef1: Deleting consensus metadata
I20260807 02:27:25.122684 28981 catalog_manager.cc:4841] TS bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173): tablet 144ea806e9074749b831602c8b4fcdfe (table test-table [id=b4b371e635be4b44b0c5e057158eb877]) successfully deleted
I20260807 02:27:25.123220 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec (DEFAULT_TABLE table=test-table [id=e1a7657c5d4e429a87de524bedff91fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.123237 29634 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec (DEFAULT_TABLE table=test-table [id=e1a7657c5d4e429a87de524bedff91fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.123519 29634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ef7e61e4746079de6cb0eedb48bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.123538 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ef7e61e4746079de6cb0eedb48bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.125550 29807 ts_tablet_manager.cc:1918] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.125728 29807 log.cc:1196] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/144ea806e9074749b831602c8b4fcdfe
I20260807 02:27:25.126179 29778 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Bootstrap starting.
I20260807 02:27:25.126425 29421 tablet_service.cc:1467] Processing CreateTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec (DEFAULT_TABLE table=test-table [id=e1a7657c5d4e429a87de524bedff91fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.126708 29421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e5ef7e61e4746079de6cb0eedb48bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.127466 29778 tablet_bootstrap.cc:654] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.128793 29778 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: No bootstrap required, opened a new log
I20260807 02:27:25.128924 29778 ts_tablet_manager.cc:1397] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 02:27:25.129549 29777 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Bootstrap starting.
I20260807 02:27:25.129710 29779 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Bootstrap starting.
I20260807 02:27:25.129846 29778 raft_consensus.cc:348] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.129962 29778 raft_consensus.cc:374] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.130002 29778 raft_consensus.cc:729] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.130192 29778 consensus_queue.cc:260] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.130540 29777 tablet_bootstrap.cc:654] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.130834 29779 tablet_bootstrap.cc:654] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.131433 29778 ts_tablet_manager.cc:1428] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:25.131878 29777 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: No bootstrap required, opened a new log
I20260807 02:27:25.131959 29777 ts_tablet_manager.cc:1397] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:25.132364 29777 raft_consensus.cc:348] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.132460 29777 raft_consensus.cc:374] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.132490 29777 raft_consensus.cc:729] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.132613 29777 consensus_queue.cc:260] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.132993 29779 tablet_bootstrap.cc:492] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: No bootstrap required, opened a new log
I20260807 02:27:25.133096 29779 ts_tablet_manager.cc:1397] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260807 02:27:25.133603 29777 ts_tablet_manager.cc:1428] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:25.133601 29779 raft_consensus.cc:348] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.133731 29779 raft_consensus.cc:374] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.133780 29779 raft_consensus.cc:729] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7a525279f74dd490df667a6a1411c0, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.133927 29779 consensus_queue.cc:260] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.134220 29779 ts_tablet_manager.cc:1428] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:25.128796 29807 ts_tablet_manager.cc:1939] T 144ea806e9074749b831602c8b4fcdfe P b9ddf9fae22f483a86be638644521f18: Deleting consensus metadata
I20260807 02:27:25.135944 29812 ts_tablet_manager.cc:1905] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.136379 28980 catalog_manager.cc:4841] TS b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799): tablet 144ea806e9074749b831602c8b4fcdfe (table test-table [id=b4b371e635be4b44b0c5e057158eb877]) successfully deleted
I20260807 02:27:25.138409 29812 ts_tablet_manager.cc:1918] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.138542 29812 log.cc:1196] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/144ea806e9074749b831602c8b4fcdfe
I20260807 02:27:25.138917 29812 ts_tablet_manager.cc:1939] T 144ea806e9074749b831602c8b4fcdfe P bf7a525279f74dd490df667a6a1411c0: Deleting consensus metadata
I20260807 02:27:25.139977 28981 catalog_manager.cc:4841] TS bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605): tablet 144ea806e9074749b831602c8b4fcdfe (table test-table [id=b4b371e635be4b44b0c5e057158eb877]) successfully deleted
I20260807 02:27:25.161751 29786 raft_consensus.cc:482] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:25.161906 29786 raft_consensus.cc:504] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.163141 29786 leader_election.cc:288] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.167749 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.163347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39173 (local address 127.17.53.1:33541)
0807 02:27:25.163628 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.163646 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.163758 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.164196 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.164214 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.164241 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.164379 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.164388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.165381 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.165385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.166378 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.166389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.167300 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.167303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.167400 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.167427 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.167506 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.167574 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
I20260807 02:27:25.167763 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.163444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:33541 (local address 127.17.53.2:39173)
0807 02:27:25.163576 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.163580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.163725 (+ 145us) server_negotiation.cc:407] Connection header received
0807 02:27:25.163918 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.163923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.163979 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.164034 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.164506 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.165266 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.166516 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.167153 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.167270 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.167343 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.167537 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.167613 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:27:25.168344 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.163924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:45181 (local address 127.17.53.3:44605)
0807 02:27:25.164038 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.164042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.164272 (+ 230us) server_negotiation.cc:407] Connection header received
0807 02:27:25.164456 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.164463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.164539 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.164588 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:25.165160 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.165930 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.167070 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.167642 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.167755 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.167836 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.168146 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.168226 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:27:25.168486 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.163770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:44605 (local address 127.17.53.1:45181)
0807 02:27:25.164193 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.164206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.164331 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.164873 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.164876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.164897 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:25.165011 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.165017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.166070 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.166074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.166949 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.166958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.167911 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.167917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.168011 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.168031 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.168104 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.168349 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":120,"threads_started":1}
I20260807 02:27:25.168689 29449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5ef7e61e4746079de6cb0eedb48bec" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1" is_pre_election: true
I20260807 02:27:25.168996 29449 raft_consensus.cc:2454] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.168999 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5ef7e61e4746079de6cb0eedb48bec" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0" is_pre_election: true
I20260807 02:27:25.169153 29667 raft_consensus.cc:2454] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.169477 29143 leader_election.cc:302] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bbd38fedd4974e19baaecd234f0aeef1; no voters:
I20260807 02:27:25.169780 29786 raft_consensus.cc:2781] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:25.169862 29786 raft_consensus.cc:482] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:25.169911 29786 raft_consensus.cc:3037] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.171103 29786 raft_consensus.cc:504] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.171486 29786 leader_election.cc:288] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Requested vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.171818 29449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5ef7e61e4746079de6cb0eedb48bec" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1"
I20260807 02:27:25.171833 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e5ef7e61e4746079de6cb0eedb48bec" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0"
I20260807 02:27:25.171942 29449 raft_consensus.cc:3037] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.171983 29667 raft_consensus.cc:3037] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.173179 29667 raft_consensus.cc:2454] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.173290 29449 raft_consensus.cc:2454] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.173481 29143 leader_election.cc:302] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.173691 29786 raft_consensus.cc:2781] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:25.173930 29786 raft_consensus.cc:686] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 1 LEADER]: Becoming Leader. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Running, Role: LEADER
I20260807 02:27:25.174172 29786 consensus_queue.cc:237] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.176595 29002 catalog_manager.cc:5495] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9ddf9fae22f483a86be638644521f18 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } health_report { overall_health: UNKNOWN } } }
W20260807 02:27:25.195416 29732 tablet.cc:2369] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:25.203024 29521 tablet.cc:2369] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:25.219314 29318 tablet.cc:2369] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069644, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.62@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:25.228967 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.218699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.17.53.62:40867)
0807 02:27:25.218855 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.218862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.219122 (+ 260us) server_negotiation.cc:407] Connection header received
0807 02:27:25.219269 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.219274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.219343 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.219465 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:25.220392 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.221071 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.222267 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.222495 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.222581 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.225353 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:25.225356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:25.225358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:25.225380 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:25.226940 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.227509 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.227512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.227513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.227592 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.227913 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.227915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.227916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.228092 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:25.228171 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.228582 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.228738 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":55}
I20260807 02:27:25.228983 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.218584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:40867 (local address 127.0.0.1:49310)
0807 02:27:25.218989 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.219006 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.219146 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.219595 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.219599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.220089 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:25.220229 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.220237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.221200 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.221204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.222125 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.222136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.222623 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.222627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.222715 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.223372 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:25.223390 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:25.225187 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:25.227060 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:25.227070 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:25.227077 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:25.227387 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:25.227699 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:25.227702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:25.227706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:25.227815 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:25.228219 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:25.228226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:25.228362 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.228492 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.228670 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":211,"threads_started":1}
I20260807 02:27:25.233632 29002 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:49310:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 02:27:25.234503 29002 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:25.240061 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 3df991f31a314049bb2db2128952e9ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dff8c9c35b554908926455b392ad7a45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:27:25.240531 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df991f31a314049bb2db2128952e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.240551 29634 tablet_service.cc:1467] Processing CreateTablet for tablet 3df991f31a314049bb2db2128952e9ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dff8c9c35b554908926455b392ad7a45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:27:25.241014 29634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df991f31a314049bb2db2128952e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.243943 29778 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: Bootstrap starting.
I20260807 02:27:25.243965 29779 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: Bootstrap starting.
I20260807 02:27:25.245020 29779 tablet_bootstrap.cc:654] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.245110 29778 tablet_bootstrap.cc:654] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.245275 29421 tablet_service.cc:1467] Processing CreateTablet for tablet 3df991f31a314049bb2db2128952e9ab (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dff8c9c35b554908926455b392ad7a45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 02:27:25.245697 29421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df991f31a314049bb2db2128952e9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.246376 29779 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: No bootstrap required, opened a new log
I20260807 02:27:25.246456 29779 ts_tablet_manager.cc:1397] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:25.246903 29779 raft_consensus.cc:348] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.247013 29779 raft_consensus.cc:374] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.247045 29779 raft_consensus.cc:729] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7a525279f74dd490df667a6a1411c0, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.247190 29779 consensus_queue.cc:260] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.247469 29779 ts_tablet_manager.cc:1428] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:27:25.247484 29785 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.247759 29785 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.248924 29777 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: Bootstrap starting.
I20260807 02:27:25.250000 29777 tablet_bootstrap.cc:654] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.250988 29778 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: No bootstrap required, opened a new log
I20260807 02:27:25.251089 29778 ts_tablet_manager.cc:1397] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 02:27:25.251451 29777 tablet_bootstrap.cc:492] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: No bootstrap required, opened a new log
I20260807 02:27:25.251542 29777 ts_tablet_manager.cc:1397] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:25.251575 29778 raft_consensus.cc:348] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.251667 29778 raft_consensus.cc:374] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.251689 29778 raft_consensus.cc:729] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.251834 29778 consensus_queue.cc:260] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.251991 29777 raft_consensus.cc:348] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.252089 29778 ts_tablet_manager.cc:1428] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:27:25.252101 29777 raft_consensus.cc:374] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.252135 29777 raft_consensus.cc:729] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.252377 29786 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.252530 29786 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.252982 29777 consensus_queue.cc:260] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.253336 29777 ts_tablet_manager.cc:1428] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:25.253319 29798 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.253484 29798 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.268006 29786 raft_consensus.cc:482] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:25.268162 29786 raft_consensus.cc:504] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.268570 29786 leader_election.cc:288] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.269090 29449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df991f31a314049bb2db2128952e9ab" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1" is_pre_election: true
I20260807 02:27:25.269291 29449 raft_consensus.cc:2454] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.269279 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df991f31a314049bb2db2128952e9ab" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0" is_pre_election: true
I20260807 02:27:25.269428 29667 raft_consensus.cc:2454] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.269695 29143 leader_election.cc:302] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bbd38fedd4974e19baaecd234f0aeef1; no voters:
I20260807 02:27:25.270002 29786 raft_consensus.cc:2781] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:25.270053 29786 raft_consensus.cc:482] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:25.270079 29786 raft_consensus.cc:3037] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.271256 29786 raft_consensus.cc:504] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.271663 29786 leader_election.cc:288] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Requested vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.272113 29449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df991f31a314049bb2db2128952e9ab" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1"
I20260807 02:27:25.272120 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df991f31a314049bb2db2128952e9ab" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0"
I20260807 02:27:25.272267 29667 raft_consensus.cc:3037] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.272277 29449 raft_consensus.cc:3037] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.273638 29449 raft_consensus.cc:2454] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.273627 29667 raft_consensus.cc:2454] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.273933 29143 leader_election.cc:302] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bbd38fedd4974e19baaecd234f0aeef1; no voters:
I20260807 02:27:25.274137 29786 raft_consensus.cc:2781] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:25.274432 29786 raft_consensus.cc:686] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [term 1 LEADER]: Becoming Leader. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Running, Role: LEADER
I20260807 02:27:25.274590 29786 consensus_queue.cc:237] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.275566 29830 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: TxnStatusTablet state changed. Reason: New leader b9ddf9fae22f483a86be638644521f18. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.275692 29830 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:27:25.276492 29843 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 02:27:25.276661 29843 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 02:27:25.277173 28999 catalog_manager.cc:5495] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9ddf9fae22f483a86be638644521f18 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:25.294384 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.289491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.17.53.1:38799)
0807 02:27:25.289666 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.289672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.289863 (+ 191us) server_negotiation.cc:407] Connection header received
0807 02:27:25.290091 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.290094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.290167 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.290276 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.290624 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.291486 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.292881 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.293109 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.293201 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.293570 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.294000 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.294089 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.294216 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.294270 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 02:27:25.294417 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.289618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.0.0.1:46348)
0807 02:27:25.289775 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.289791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.289912 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.290342 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.290346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.290359 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:27:25.290479 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.290486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.291651 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.291663 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.292680 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.292704 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.293262 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.293266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.293360 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.293414 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.294097 (+ 683us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.294113 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.294189 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.294275 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 02:27:25.298138 29667 raft_consensus.cc:1261] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.298324 29449 raft_consensus.cc:1261] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.298656 29786 consensus_queue.cc:1035] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.298899 29830 consensus_queue.cc:1035] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.300334 29785 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: TxnStatusTablet state changed. Reason: New leader b9ddf9fae22f483a86be638644521f18. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.300455 29785 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.300655 29798 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: TxnStatusTablet state changed. Reason: New leader b9ddf9fae22f483a86be638644521f18. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.301326 29798 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.302490 29786 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.302600 29786 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:27:25.302714 29793 mvcc.cc:205] Tried to move back new op lower bound from 7315741267135090688 to 7315741267046301696. Current Snapshot: MvccSnapshot[applied={T|T < 7315741267135090688}]
I20260807 02:27:25.304556 29785 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.304669 29785 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bf7a525279f74dd490df667a6a1411c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.305596 29830 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.305711 29830 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P b9ddf9fae22f483a86be638644521f18: This TxnStatusTablet replica's current role is: LEADER
I20260807 02:27:25.306535 29792 mvcc.cc:205] Tried to move back new op lower bound from 7315741267135090688 to 7315741267046301696. Current Snapshot: MvccSnapshot[applied={T|T < 7315741267135090688}]
I20260807 02:27:25.306726 29798 tablet_replica.cc:439] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } } }
I20260807 02:27:25.306818 29798 tablet_replica.cc:442] T 3df991f31a314049bb2db2128952e9ab P bbd38fedd4974e19baaecd234f0aeef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 02:27:25.307500 29794 mvcc.cc:205] Tried to move back new op lower bound from 7315741267135090688 to 7315741267046301696. Current Snapshot: MvccSnapshot[applied={T|T < 7315741267135090688 or (T in {7315741267135090688})}]
I20260807 02:27:25.311404 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.307700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.17.53.1:38799)
0807 02:27:25.307821 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.307826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.307841 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:25.307885 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.307890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.307938 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.308319 (+ 381us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.308412 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.309147 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.310277 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.310489 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.310621 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.310866 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.310941 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.311021 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.311114 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.311165 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 02:27:25.322753 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.318884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:40319 (local address 127.17.53.1:38799)
0807 02:27:25.319009 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.319013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.319223 (+ 210us) server_negotiation.cc:407] Connection header received
0807 02:27:25.319309 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.319312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.319355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.319428 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.319741 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.320270 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.321637 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.321879 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.321971 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.322314 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.322380 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.322438 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.322574 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.322623 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53}
I20260807 02:27:25.322809 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.318803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.17.53.1:40319)
0807 02:27:25.319097 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.319111 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.319228 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.319461 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.319464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.319472 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:27:25.319618 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.319624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.320383 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.320387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.321503 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.321515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.321999 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.322003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.322120 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.322176 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.322472 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.322479 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.322545 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.322628 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260807 02:27:25.329914 29667 raft_consensus.cc:1261] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.329993 29449 raft_consensus.cc:1261] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.330343 29786 consensus_queue.cc:1035] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.330478 29830 consensus_queue.cc:1035] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.374735 29003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.374925 29003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.377736 29003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Sending DeleteTablet for 3 replicas of tablet 9e5ef7e61e4746079de6cb0eedb48bec
I20260807 02:27:25.378319 29215 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260807 02:27:25.378513 29807 tablet_replica.cc:330] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: stopping tablet replica
I20260807 02:27:25.378621 29807 raft_consensus.cc:2229] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:25.378619 29421 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60496
I20260807 02:27:25.378758 29634 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e5ef7e61e4746079de6cb0eedb48bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 02:27:25.378831 29805 tablet_replica.cc:330] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: stopping tablet replica
I20260807 02:27:25.378904 29807 raft_consensus.cc:2258] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.378969 29805 raft_consensus.cc:2229] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.379093 29805 raft_consensus.cc:2258] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.379278 29812 tablet_replica.cc:330] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: stopping tablet replica
I20260807 02:27:25.379386 29812 raft_consensus.cc:2229] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.379496 29812 raft_consensus.cc:2258] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.379864 29807 ts_tablet_manager.cc:1905] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.380376 29812 ts_tablet_manager.cc:1905] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.382740 29807 ts_tablet_manager.cc:1918] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:27:25.382825 29807 log.cc:1196] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/9e5ef7e61e4746079de6cb0eedb48bec
I20260807 02:27:25.382912 29812 ts_tablet_manager.cc:1918] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:27:25.383002 29812 log.cc:1196] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/9e5ef7e61e4746079de6cb0eedb48bec
I20260807 02:27:25.383083 29807 ts_tablet_manager.cc:1939] T 9e5ef7e61e4746079de6cb0eedb48bec P b9ddf9fae22f483a86be638644521f18: Deleting consensus metadata
I20260807 02:27:25.383303 29812 ts_tablet_manager.cc:1939] T 9e5ef7e61e4746079de6cb0eedb48bec P bf7a525279f74dd490df667a6a1411c0: Deleting consensus metadata
I20260807 02:27:25.383737 28980 catalog_manager.cc:4841] TS b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799): tablet 9e5ef7e61e4746079de6cb0eedb48bec (table test-table [id=e1a7657c5d4e429a87de524bedff91fb]) successfully deleted
I20260807 02:27:25.384413 29805 ts_tablet_manager.cc:1905] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.387055 28981 catalog_manager.cc:4841] TS bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605): tablet 9e5ef7e61e4746079de6cb0eedb48bec (table test-table [id=e1a7657c5d4e429a87de524bedff91fb]) successfully deleted
I20260807 02:27:25.387216 29805 ts_tablet_manager.cc:1918] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:27:25.387295 29805 log.cc:1196] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/9e5ef7e61e4746079de6cb0eedb48bec
I20260807 02:27:25.387547 29805 ts_tablet_manager.cc:1939] T 9e5ef7e61e4746079de6cb0eedb48bec P bbd38fedd4974e19baaecd234f0aeef1: Deleting consensus metadata
I20260807 02:27:25.388303 28981 catalog_manager.cc:4841] TS bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173): tablet 9e5ef7e61e4746079de6cb0eedb48bec (table test-table [id=e1a7657c5d4e429a87de524bedff91fb]) successfully deleted
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[28934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 02:27:25.416038 29082 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:25.406989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.17.53.62:40867)
0807 02:27:25.407295 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.407301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.407433 (+ 132us) server_negotiation.cc:407] Connection header received
0807 02:27:25.407614 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.407617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.407678 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.407779 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:25.408556 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.409456 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.410742 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.411358 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.411481 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.415891 (+ 4410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49320: BlockingRecv error: recv got EOF from 127.0.0.1:49320 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[28934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM
I20260807 02:27:25.426636 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.417678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.17.53.62:40867)
0807 02:27:25.417803 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.417807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.418221 (+ 414us) server_negotiation.cc:407] Connection header received
0807 02:27:25.418384 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.418387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.418438 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.418522 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:25.419149 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.419650 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.420451 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.420719 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.420882 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.423386 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:25.423396 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:25.423399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:25.423421 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:25.424940 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.425361 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.425365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.425367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.425454 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:25.425769 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:25.425775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:25.425777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:25.425958 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:25.426098 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.426357 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.426470 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:27:25.429793 29003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:25.430248 29003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:25.435690 29634 tablet_service.cc:1467] Processing CreateTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 (DEFAULT_TABLE table=test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.435742 29215 tablet_service.cc:1467] Processing CreateTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 (DEFAULT_TABLE table=test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.436070 29634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60fc31edc0b494aa30cd6cad330b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.436082 29215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60fc31edc0b494aa30cd6cad330b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.437084 29421 tablet_service.cc:1467] Processing CreateTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 (DEFAULT_TABLE table=test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.437407 29421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c60fc31edc0b494aa30cd6cad330b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.438421 29779 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Bootstrap starting.
I20260807 02:27:25.438715 29778 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Bootstrap starting.
I20260807 02:27:25.439429 29779 tablet_bootstrap.cc:654] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.439555 29778 tablet_bootstrap.cc:654] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.440673 29777 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Bootstrap starting.
I20260807 02:27:25.440752 29778 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: No bootstrap required, opened a new log
I20260807 02:27:25.440831 29779 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: No bootstrap required, opened a new log
I20260807 02:27:25.440846 29778 ts_tablet_manager.cc:1397] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:25.440907 29779 ts_tablet_manager.cc:1397] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:25.441242 29778 raft_consensus.cc:348] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.441365 29778 raft_consensus.cc:374] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.441393 29778 raft_consensus.cc:729] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.441334 29779 raft_consensus.cc:348] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.441439 29779 raft_consensus.cc:374] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.441483 29779 raft_consensus.cc:729] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7a525279f74dd490df667a6a1411c0, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.441500 29778 consensus_queue.cc:260] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.441581 29777 tablet_bootstrap.cc:654] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.441627 29779 consensus_queue.cc:260] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.441751 29778 ts_tablet_manager.cc:1428] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 02:27:25.441932 29779 ts_tablet_manager.cc:1428] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:25.444751 29777 tablet_bootstrap.cc:492] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: No bootstrap required, opened a new log
I20260807 02:27:25.444882 29777 ts_tablet_manager.cc:1397] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 02:27:25.445338 29777 raft_consensus.cc:348] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.445482 29777 raft_consensus.cc:374] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.445526 29777 raft_consensus.cc:729] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.445665 29777 consensus_queue.cc:260] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.445966 29777 ts_tablet_manager.cc:1428] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:27:25.497478 29852 raft_consensus.cc:482] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:25.497632 29852 raft_consensus.cc:504] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.498072 29852 leader_election.cc:288] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.498597 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60fc31edc0b494aa30cd6cad330b5a1" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0" is_pre_election: true
I20260807 02:27:25.498634 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60fc31edc0b494aa30cd6cad330b5a1" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1" is_pre_election: true
I20260807 02:27:25.498770 29667 raft_consensus.cc:2454] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.498823 29448 raft_consensus.cc:2454] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 0.
I20260807 02:27:25.499111 29143 leader_election.cc:302] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bbd38fedd4974e19baaecd234f0aeef1; no voters:
I20260807 02:27:25.499308 29852 raft_consensus.cc:2781] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:25.499388 29852 raft_consensus.cc:482] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:25.499442 29852 raft_consensus.cc:3037] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.500579 29852 raft_consensus.cc:504] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.500972 29852 leader_election.cc:288] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Requested vote from peers bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.501403 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60fc31edc0b494aa30cd6cad330b5a1" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0"
I20260807 02:27:25.501410 29448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c60fc31edc0b494aa30cd6cad330b5a1" candidate_uuid: "b9ddf9fae22f483a86be638644521f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd38fedd4974e19baaecd234f0aeef1"
I20260807 02:27:25.501534 29448 raft_consensus.cc:3037] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.501533 29667 raft_consensus.cc:3037] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.502432 29667 raft_consensus.cc:2454] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.502738 29143 leader_election.cc:302] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9ddf9fae22f483a86be638644521f18, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.502784 29448 raft_consensus.cc:2454] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ddf9fae22f483a86be638644521f18 in term 1.
I20260807 02:27:25.502943 29852 raft_consensus.cc:2781] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:25.503034 29852 raft_consensus.cc:686] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 1 LEADER]: Becoming Leader. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Running, Role: LEADER
I20260807 02:27:25.503178 29852 consensus_queue.cc:237] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.505072 29003 catalog_manager.cc:5495] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9ddf9fae22f483a86be638644521f18 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "b9ddf9fae22f483a86be638644521f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:25.527395 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.524078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.17.53.1:38799)
0807 02:27:25.524214 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.524218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.524233 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:25.524338 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.524341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.524395 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.524478 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.524798 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.525413 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.526330 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.526566 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.526680 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.526950 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.527029 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.527110 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.527204 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.527253 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 02:27:25.529513 29448 raft_consensus.cc:1261] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.529522 29667 raft_consensus.cc:1261] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Refusing update from remote peer b9ddf9fae22f483a86be638644521f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.530045 29852 consensus_queue.cc:1035] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.530303 29857 consensus_queue.cc:1035] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.538401 29003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.538545 29003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.541155 29003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Sending DeleteTablet for 3 replicas of tablet c60fc31edc0b494aa30cd6cad330b5a1
I20260807 02:27:25.541847 29215 tablet_service.cc:1514] Processing DeleteTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260807 02:27:25.541901 29634 tablet_service.cc:1514] Processing DeleteTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 02:27:25.541887 29421 tablet_service.cc:1514] Processing DeleteTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60496
I20260807 02:27:25.542033 29807 tablet_replica.cc:330] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: stopping tablet replica
I20260807 02:27:25.542042 29812 tablet_replica.cc:330] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: stopping tablet replica
I20260807 02:27:25.542057 29805 tablet_replica.cc:330] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: stopping tablet replica
I20260807 02:27:25.542141 29812 raft_consensus.cc:2229] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.542141 29807 raft_consensus.cc:2229] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:25.542178 29805 raft_consensus.cc:2229] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.542256 29812 raft_consensus.cc:2258] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.542303 29805 raft_consensus.cc:2258] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.542320 29807 raft_consensus.cc:2258] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.543079 29805 ts_tablet_manager.cc:1905] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.543087 29807 ts_tablet_manager.cc:1905] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.545794 29805 ts_tablet_manager.cc:1918] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.545830 29807 ts_tablet_manager.cc:1918] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.545889 29805 log.cc:1196] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/c60fc31edc0b494aa30cd6cad330b5a1
I20260807 02:27:25.545903 29807 log.cc:1196] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/c60fc31edc0b494aa30cd6cad330b5a1
I20260807 02:27:25.546175 29805 ts_tablet_manager.cc:1939] T c60fc31edc0b494aa30cd6cad330b5a1 P bbd38fedd4974e19baaecd234f0aeef1: Deleting consensus metadata
I20260807 02:27:25.546175 29807 ts_tablet_manager.cc:1939] T c60fc31edc0b494aa30cd6cad330b5a1 P b9ddf9fae22f483a86be638644521f18: Deleting consensus metadata
I20260807 02:27:25.546783 29634 tablet_service.cc:1514] Processing DeleteTablet for tablet c60fc31edc0b494aa30cd6cad330b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 02:27:25.546931 28981 catalog_manager.cc:4841] TS bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173): tablet c60fc31edc0b494aa30cd6cad330b5a1 (table test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]) successfully deleted
I20260807 02:27:25.547294 29812 ts_tablet_manager.cc:1905] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.547472 28980 catalog_manager.cc:4841] TS b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799): tablet c60fc31edc0b494aa30cd6cad330b5a1 (table test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]) successfully deleted
I20260807 02:27:25.547634 29003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:25.548156 29003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:25.549667 29812 ts_tablet_manager.cc:1918] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:25.549808 29812 log.cc:1196] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/c60fc31edc0b494aa30cd6cad330b5a1
I20260807 02:27:25.550267 29812 ts_tablet_manager.cc:1939] T c60fc31edc0b494aa30cd6cad330b5a1 P bf7a525279f74dd490df667a6a1411c0: Deleting consensus metadata
I20260807 02:27:25.552153 28981 catalog_manager.cc:4841] TS bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605): tablet c60fc31edc0b494aa30cd6cad330b5a1 (table test-table [id=ef601ef21c4d477bb3fc432bfb3eca5e]) successfully deleted
W20260807 02:27:25.552505 28981 catalog_manager.cc:4808] TS bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605): delete failed for tablet c60fc31edc0b494aa30cd6cad330b5a1 because the tablet was not found. No further retry: Not found: Tablet not found: c60fc31edc0b494aa30cd6cad330b5a1
I20260807 02:27:25.553542 29215 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccfe651af914e7bab2ae953de091a33 (DEFAULT_TABLE table=test-table [id=41e2f69f6ce94033bf78358c2961d558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.553757 29421 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccfe651af914e7bab2ae953de091a33 (DEFAULT_TABLE table=test-table [id=41e2f69f6ce94033bf78358c2961d558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.553871 29215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccfe651af914e7bab2ae953de091a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.554023 29421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccfe651af914e7bab2ae953de091a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.555884 29778 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: Bootstrap starting.
I20260807 02:27:25.556282 29777 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Bootstrap starting.
I20260807 02:27:25.556842 29778 tablet_bootstrap.cc:654] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.557206 29777 tablet_bootstrap.cc:654] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.557559 29634 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccfe651af914e7bab2ae953de091a33 (DEFAULT_TABLE table=test-table [id=41e2f69f6ce94033bf78358c2961d558]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:25.557852 29634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccfe651af914e7bab2ae953de091a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:25.558542 29777 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: No bootstrap required, opened a new log
I20260807 02:27:25.558619 29777 ts_tablet_manager.cc:1397] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:25.559037 29777 raft_consensus.cc:348] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.559168 29777 raft_consensus.cc:374] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.559206 29777 raft_consensus.cc:729] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.559372 29777 consensus_queue.cc:260] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.559664 29777 ts_tablet_manager.cc:1428] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:25.560590 29779 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Bootstrap starting.
I20260807 02:27:25.561648 29779 tablet_bootstrap.cc:654] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:25.562800 29779 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: No bootstrap required, opened a new log
I20260807 02:27:25.562885 29779 ts_tablet_manager.cc:1397] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:25.563184 29778 tablet_bootstrap.cc:492] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: No bootstrap required, opened a new log
I20260807 02:27:25.563297 29778 ts_tablet_manager.cc:1397] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260807 02:27:25.563299 29779 raft_consensus.cc:348] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.563413 29779 raft_consensus.cc:374] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.563447 29779 raft_consensus.cc:729] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7a525279f74dd490df667a6a1411c0, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.563561 29779 consensus_queue.cc:260] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.563793 29779 ts_tablet_manager.cc:1428] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:27:25.563784 29778 raft_consensus.cc:348] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.563928 29778 raft_consensus.cc:374] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:25.563984 29778 raft_consensus.cc:729] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ddf9fae22f483a86be638644521f18, State: Initialized, Role: FOLLOWER
I20260807 02:27:25.564136 29778 consensus_queue.cc:260] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.564458 29778 ts_tablet_manager.cc:1428] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:27:25.602878 29798 raft_consensus.cc:482] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:25.603060 29798 raft_consensus.cc:504] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.603577 29798 leader_election.cc:288] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.604022 29249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccfe651af914e7bab2ae953de091a33" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ddf9fae22f483a86be638644521f18" is_pre_election: true
I20260807 02:27:25.604039 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccfe651af914e7bab2ae953de091a33" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0" is_pre_election: true
I20260807 02:27:25.604207 29667 raft_consensus.cc:2454] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 0.
I20260807 02:27:25.604207 29249 raft_consensus.cc:2454] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 0.
I20260807 02:27:25.604532 29354 leader_election.cc:302] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd38fedd4974e19baaecd234f0aeef1, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.604815 29798 raft_consensus.cc:2781] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:25.604887 29798 raft_consensus.cc:482] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:25.604952 29798 raft_consensus.cc:3037] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.606123 29798 raft_consensus.cc:504] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.606575 29798 leader_election.cc:288] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 election: Requested vote from peers b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799), bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605)
I20260807 02:27:25.606999 29667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccfe651af914e7bab2ae953de091a33" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf7a525279f74dd490df667a6a1411c0"
I20260807 02:27:25.607164 29667 raft_consensus.cc:3037] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.608526 29667 raft_consensus.cc:2454] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 1.
I20260807 02:27:25.608915 29249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccfe651af914e7bab2ae953de091a33" candidate_uuid: "bbd38fedd4974e19baaecd234f0aeef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ddf9fae22f483a86be638644521f18"
I20260807 02:27:25.608997 29354 leader_election.cc:302] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbd38fedd4974e19baaecd234f0aeef1, bf7a525279f74dd490df667a6a1411c0; no voters:
I20260807 02:27:25.609071 29249 raft_consensus.cc:3037] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:25.609200 29798 raft_consensus.cc:2781] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:25.609382 29798 raft_consensus.cc:686] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 LEADER]: Becoming Leader. State: Replica: bbd38fedd4974e19baaecd234f0aeef1, State: Running, Role: LEADER
I20260807 02:27:25.609552 29798 consensus_queue.cc:237] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } }
I20260807 02:27:25.610373 29249 raft_consensus.cc:2454] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd38fedd4974e19baaecd234f0aeef1 in term 1.
I20260807 02:27:25.611596 29002 catalog_manager.cc:5495] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "bbd38fedd4974e19baaecd234f0aeef1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbd38fedd4974e19baaecd234f0aeef1" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 } health_report { overall_health: UNKNOWN } } }
I20260807 02:27:25.633136 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.629910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.17.53.2:39173)
0807 02:27:25.630047 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.630054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.630073 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:27:25.630262 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.630265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.630306 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.630390 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.630655 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.631160 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.632038 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.632272 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.632390 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.632663 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.632738 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.632836 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.632935 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.632986 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:27:25.642860 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.639838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:42917 (local address 127.17.53.1:38799)
0807 02:27:25.639985 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.639990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.640153 (+ 163us) server_negotiation.cc:407] Connection header received
0807 02:27:25.640237 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.640240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.640300 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.640363 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.640554 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.641087 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.641957 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.642161 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.642240 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.642490 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.642556 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.642608 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.642709 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.642748 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:27:25.642990 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.639731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.17.53.2:42917)
0807 02:27:25.640069 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.640082 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.640168 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.640386 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.640388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.640394 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:27:25.640486 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.640490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.641180 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.641183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.641855 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.641865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.642264 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.642266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.642329 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.642372 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.642632 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.642637 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.642685 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.642846 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":166,"threads_started":1}
I20260807 02:27:25.649529 29249 raft_consensus.cc:1261] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Refusing update from remote peer bbd38fedd4974e19baaecd234f0aeef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.649554 29667 raft_consensus.cc:1261] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Refusing update from remote peer bbd38fedd4974e19baaecd234f0aeef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:25.650136 29886 consensus_queue.cc:1035] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ddf9fae22f483a86be638644521f18" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.650349 29798 consensus_queue.cc:1035] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf7a525279f74dd490df667a6a1411c0" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:25.667229 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.664001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:59081 (local address 127.17.53.2:39173)
0807 02:27:25.664119 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:25.664124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:25.664176 (+ 52us) server_negotiation.cc:407] Connection header received
0807 02:27:25.664350 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:25.664353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:25.664409 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:25.664474 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:25.664831 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.665355 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.666207 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.666432 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.666506 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.666768 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.666853 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.666939 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:25.667031 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 02:27:25.667082 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:27:25.667229 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.663958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39173 (local address 127.17.53.1:59081)
0807 02:27:25.664105 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.664123 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:25.664228 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:25.664501 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:25.664503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:25.664510 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 02:27:25.664606 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.664610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.665463 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.665466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.666104 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:25.666114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:25.666523 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:25.666527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:25.666611 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:25.666641 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 02:27:25.666944 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 02:27:25.666954 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 02:27:25.667021 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 02:27:25.667108 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 02:27:25.689479 29002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.689631 29002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:25.691944 29002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c567405852b44e96b39d2eb05dce5ca4: Sending DeleteTablet for 3 replicas of tablet 0ccfe651af914e7bab2ae953de091a33
I20260807 02:27:25.692495 29215 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccfe651af914e7bab2ae953de091a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260807 02:27:25.692507 29421 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccfe651af914e7bab2ae953de091a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:60496
I20260807 02:27:25.692696 29807 tablet_replica.cc:330] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: stopping tablet replica
I20260807 02:27:25.692696 29805 tablet_replica.cc:330] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: stopping tablet replica
I20260807 02:27:25.692727 29634 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccfe651af914e7bab2ae953de091a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:25 UTC) from {username='oryx', principal='oryx/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 02:27:25.692878 29805 raft_consensus.cc:2229] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:25.692874 29807 raft_consensus.cc:2229] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.692924 29812 tablet_replica.cc:330] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: stopping tablet replica
I20260807 02:27:25.692996 29807 raft_consensus.cc:2258] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.693061 29812 raft_consensus.cc:2229] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:25.693086 29805 raft_consensus.cc:2258] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.693185 29812 raft_consensus.cc:2258] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:25.693606 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 29116
I20260807 02:27:25.694015 29812 ts_tablet_manager.cc:1905] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.694068 29807 ts_tablet_manager.cc:1905] T 0ccfe651af914e7bab2ae953de091a33 P b9ddf9fae22f483a86be638644521f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.697592 29812 ts_tablet_manager.cc:1918] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:27:25.697690 29812 log.cc:1196] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/0ccfe651af914e7bab2ae953de091a33
I20260807 02:27:25.697983 29812 ts_tablet_manager.cc:1939] T 0ccfe651af914e7bab2ae953de091a33 P bf7a525279f74dd490df667a6a1411c0: Deleting consensus metadata
I20260807 02:27:25.698781 28981 catalog_manager.cc:4841] TS bf7a525279f74dd490df667a6a1411c0 (127.17.53.3:44605): tablet 0ccfe651af914e7bab2ae953de091a33 (table test-table [id=41e2f69f6ce94033bf78358c2961d558]) successfully deleted
I20260807 02:27:25.705209 29805 ts_tablet_manager.cc:1905] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:25.707756 29805 ts_tablet_manager.cc:1918] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 02:27:25.707861 29805 log.cc:1196] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/0ccfe651af914e7bab2ae953de091a33
I20260807 02:27:25.708105 29805 ts_tablet_manager.cc:1939] T 0ccfe651af914e7bab2ae953de091a33 P bbd38fedd4974e19baaecd234f0aeef1: Deleting consensus metadata
I20260807 02:27:25.708912 28981 catalog_manager.cc:4841] TS bbd38fedd4974e19baaecd234f0aeef1 (127.17.53.2:39173): tablet 0ccfe651af914e7bab2ae953de091a33 (table test-table [id=41e2f69f6ce94033bf78358c2961d558]) successfully deleted
I20260807 02:27:25.713120 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 29329
W20260807 02:27:25.713114 28980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:38799 (error 108)
I20260807 02:27:25.720386 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:25.719287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38799 (local address 127.0.0.1:46386)
0807 02:27:25.720215 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:25.720294 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":753,"threads_started":1}
W20260807 02:27:25.720541 28980 catalog_manager.cc:4584] TS b9ddf9fae22f483a86be638644521f18 (127.17.53.1:38799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ccfe651af914e7bab2ae953de091a33: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38799: connect: Connection refused (error 111)
I20260807 02:27:25.732910 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 29533
I20260807 02:27:25.745761 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 28956
2026-08-07T02:27:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3159 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[29902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[29902](info): set up 2 sockets
Aug 07 02:27:25 dist-test-slave-g170 krb5kdc[29902](info): commencing operation
krb5kdc: starting...
W20260807 02:27:27.070844 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:27 dist-test-slave-g170 krb5kdc[29902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:27.205703 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35357
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:27.320170 29929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:27.320472 29929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:27.320545 29929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:27.324481 29929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:27.324568 29929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:27.324606 29929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:27.324662 29929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:27.324718 29929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:27.330063 29929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:27.331147 29929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:27.332175 29929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:27.341144 29936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:27.341144 29938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:27.341477 29929 server_base.cc:1034] running on GCE node
W20260807 02:27:27.341180 29934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:27.342262 29929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:27.343456 29929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:27.344641 29929 hybrid_clock.cc:648] HybridClock initialized: now 1786069647344612 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T02:27:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1599 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:27 dist-test-slave-g170 krb5kdc[29944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:27 dist-test-slave-g170 krb5kdc[29944](info): set up 2 sockets
Aug 07 02:27:27 dist-test-slave-g170 krb5kdc[29944](info): commencing operation
krb5kdc: starting...
W20260807 02:27:28.592432 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:28 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:28.727888 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36375
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:28.843919 29959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:28.844257 29959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:28.844347 29959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:28.848474 29959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:28.848584 29959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:28.848625 29959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:28.848680 29959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:28.848718 29959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:28.854378 29959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:28.855501 29959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:28.856585 29959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:28.865542 29965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:28.865681 29970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:28.865582 29966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:28.865808 29959 server_base.cc:1034] running on GCE node
I20260807 02:27:28.866577 29959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:28.867743 29959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:28.868908 29959 hybrid_clock.cc:648] HybridClock initialized: now 1786069648868896 us; error 26 us; skew 500 ppm
Aug 07 02:27:28 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069648, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:28.871723 29959 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:28.872668 29959 webserver.cc:460] Webserver started at http://127.17.53.62:33121/ using document root <none> and password file <none>
I20260807 02:27:28.873279 29959 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:28.873345 29959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:28.873517 29959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:28.875545 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "ff34027892d74eb792320fa246a7458a"
format_stamp: "Formatted at 2026-08-07 02:27:28 on dist-test-slave-g170"
server_key: "b0d4ac4315f36620e8a5f769847f6c80"
server_key_iv: "86e51d934f74389b35140585da37d423"
server_key_version: "encryptionkey@0"
I20260807 02:27:28.876336 29959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "ff34027892d74eb792320fa246a7458a"
format_stamp: "Formatted at 2026-08-07 02:27:28 on dist-test-slave-g170"
server_key: "b0d4ac4315f36620e8a5f769847f6c80"
server_key_iv: "86e51d934f74389b35140585da37d423"
server_key_version: "encryptionkey@0"
I20260807 02:27:28.880287 29959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:27:28.882835 29978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:28.884049 29959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:27:28.884200 29959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "ff34027892d74eb792320fa246a7458a"
format_stamp: "Formatted at 2026-08-07 02:27:28 on dist-test-slave-g170"
server_key: "b0d4ac4315f36620e8a5f769847f6c80"
server_key_iv: "86e51d934f74389b35140585da37d423"
server_key_version: "encryptionkey@0"
I20260807 02:27:28.884316 29959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:28.900735 29959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:28.903818 29959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:28.904053 29959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:28.911728 29959 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36375
I20260807 02:27:28.911779 30057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36375 every 8 connection(s)
I20260807 02:27:28.912751 29959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:28.915661 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 29959
I20260807 02:27:28.915813 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:28.915911 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:28.916150 17620 external_mini_cluster.cc:1428] Setting key 9afe86693fd94c0ac28fdd43ae5546aa
I20260807 02:27:28.923513 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Bootstrap starting.
Aug 07 02:27:28 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:28.926350 30058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:28.927330 30058 log.cc:824] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:28.929371 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: No bootstrap required, opened a new log
I20260807 02:27:28.931140 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:28.917632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.17.53.62:36375)
0807 02:27:28.918042 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:28.918061 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:28.918125 (+ 64us) server_negotiation.cc:407] Connection header received
0807 02:27:28.918923 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:28.918943 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:28.919273 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:28.919555 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:28.920508 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:28.921576 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:28.922788 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:28.923258 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:28.923375 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:28.926166 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:28.926171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:28.926185 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:28.926212 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:28.928027 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:28.928509 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:28.928515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:28.928519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:28.928634 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:28.928967 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:28.928971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:28.928973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:28.929396 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:28.929604 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:28.929911 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 02:27:28.930203 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260807 02:27:28.932598 30058 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:28.932914 30058 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:28.933001 30058 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34027892d74eb792320fa246a7458a, State: Initialized, Role: FOLLOWER
I20260807 02:27:28.933519 30058 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:28.933686 30058 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:28.933751 30058 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:28.933866 30058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:28.934814 30058 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:28.935171 30058 leader_election.cc:302] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff34027892d74eb792320fa246a7458a; no voters:
I20260807 02:27:28.935523 30058 leader_election.cc:288] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:28.935645 30063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:28.935889 30063 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 LEADER]: Becoming Leader. State: Replica: ff34027892d74eb792320fa246a7458a, State: Running, Role: LEADER
I20260807 02:27:28.936244 30063 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:28.936836 30058 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:28.938027 30065 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff34027892d74eb792320fa246a7458a. Latest consensus state: current_term: 1 leader_uuid: "ff34027892d74eb792320fa246a7458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } } }
I20260807 02:27:28.938171 30065 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:28.938639 30076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:28.938598 30064 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff34027892d74eb792320fa246a7458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } } }
I20260807 02:27:28.938772 30064 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:28.942045 30076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:28.947240 30076 catalog_manager.cc:1347] Generated new cluster ID: 893a77738d1542eba4f006c151b116d3
I20260807 02:27:28.947307 30076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:28.952172 30076 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:28.953234 30076 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:28.969089 30076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Generated new TSK 0
I20260807 02:27:28.970036 30076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:29.008318 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:29.132750 30095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:29.133098 30095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:29.133193 30095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:29.137652 30095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:29.137742 30095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:29.137836 30095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:29.143425 30095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:29.144632 30095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:29.146067 30095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:29.156611 30105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:29.156747 30102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:29.156628 30107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:29.157260 30095 server_base.cc:1034] running on GCE node
I20260807 02:27:29.157738 30095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:29.158505 30095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:29.159693 30095 hybrid_clock.cc:648] HybridClock initialized: now 1786069649159673 us; error 33 us; skew 500 ppm
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:29.163316 30095 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:29.164399 30095 webserver.cc:460] Webserver started at http://127.17.53.1:36147/ using document root <none> and password file <none>
I20260807 02:27:29.165148 30095 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:29.165264 30095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:29.165537 30095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:29.167935 30095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "a5d3c535fbfa4f53be710b192620f1b2"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "8208219ef470153407eca81db056872a"
server_key_iv: "df0017e576d5906de16ea2a588f1cfcf"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.168584 30095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "a5d3c535fbfa4f53be710b192620f1b2"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "8208219ef470153407eca81db056872a"
server_key_iv: "df0017e576d5906de16ea2a588f1cfcf"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.172930 30095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 02:27:29.175644 30117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.177296 30095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:29.177459 30095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "a5d3c535fbfa4f53be710b192620f1b2"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "8208219ef470153407eca81db056872a"
server_key_iv: "df0017e576d5906de16ea2a588f1cfcf"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.177594 30095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:29.193142 30095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:29.197197 30095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:29.197444 30095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:29.198182 30095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:29.199160 30095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:29.199239 30095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.199308 30095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:29.199361 30095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.210227 30095 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45961
I20260807 02:27:29.210268 30282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45961 every 8 connection(s)
I20260807 02:27:29.211277 30095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:29.217098 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 30095
I20260807 02:27:29.217298 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:29.217669 17620 external_mini_cluster.cc:1428] Setting key a8220bb4de5a3f1e2dc682379a7cad00
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:29.229529 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.213742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:32999 (local address 127.17.53.62:36375)
0807 02:27:29.213927 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:29.213931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:29.215125 (+ 1194us) server_negotiation.cc:407] Connection header received
0807 02:27:29.216238 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:29.216242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:29.216310 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:29.216405 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:29.218335 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.219110 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.221109 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.221415 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.221523 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.224672 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:29.224680 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:29.224686 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:29.224726 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:29.226602 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.227450 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.227454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.227456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.227556 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.228066 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.228070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.228071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.228296 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:29.228389 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:29.229246 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 02:27:29.229378 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:27:29.231000 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.214171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:32999)
0807 02:27:29.214941 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:29.214997 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:29.215976 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:29.216635 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:29.216643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:29.217325 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:29.218156 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.218176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.219264 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.219268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.220934 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.220952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.221612 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.221619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.221909 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.222783 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:29.222804 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:29.224456 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:29.226754 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.226766 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.226790 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.227289 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.227691 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.227694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.227697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.227939 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.228432 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:29.228442 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:29.228915 (+ 473us) client_negotiation.cc:769] Sending connection context
0807 02:27:29.229202 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 02:27:29.229687 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":132,"threads_started":1}
I20260807 02:27:29.232652 30283 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36375
I20260807 02:27:29.233068 30283 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:29.233736 30283 heartbeater.cc:511] Master 127.17.53.62:36375 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:27:29.235682 30006 ts_manager.cc:194] Registered new tserver with Master: a5d3c535fbfa4f53be710b192620f1b2 (127.17.53.1:45961)
I20260807 02:27:29.237164 30006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:32999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:29.251892 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:29.367343 30291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:29.367619 30291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:29.367707 30291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:29.371712 30291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:29.371791 30291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:29.371877 30291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:29.376984 30291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:29.377996 30291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:29.379019 30291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:27:29.389081 30291 server_base.cc:1034] running on GCE node
W20260807 02:27:29.389099 30298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:29.389009 30300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:29.389029 30297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:29.389668 30291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:29.390350 30291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:29.391538 30291 hybrid_clock.cc:648] HybridClock initialized: now 1786069649391521 us; error 28 us; skew 500 ppm
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:29.394413 30291 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:29.395288 30291 webserver.cc:460] Webserver started at http://127.17.53.2:33097/ using document root <none> and password file <none>
I20260807 02:27:29.395853 30291 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:29.395946 30291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:29.396176 30291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:29.398245 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "080dff63567e4ca3948829f38395a252"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "54880d0fe5ce649f4639fe3996e578a3"
server_key_iv: "165be267e20019ab477253cef5778b77"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.398839 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "080dff63567e4ca3948829f38395a252"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "54880d0fe5ce649f4639fe3996e578a3"
server_key_iv: "165be267e20019ab477253cef5778b77"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.402504 30291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 02:27:29.404901 30314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.406068 30291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:29.406201 30291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "080dff63567e4ca3948829f38395a252"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "54880d0fe5ce649f4639fe3996e578a3"
server_key_iv: "165be267e20019ab477253cef5778b77"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.406309 30291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:29.425367 30291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:29.429414 30291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:29.429617 30291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:29.430256 30291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:29.431089 30291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:29.431146 30291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.431214 30291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:29.431253 30291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.440812 30291 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40241
I20260807 02:27:29.440892 30491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40241 every 8 connection(s)
I20260807 02:27:29.441789 30291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:29.449870 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 30291
I20260807 02:27:29.450023 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:29.450317 17620 external_mini_cluster.cc:1428] Setting key 7ea22725cfe44eb56c13d413bccf5289
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:29.456871 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.443674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:55359 (local address 127.17.53.62:36375)
0807 02:27:29.443814 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:29.443818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:29.444956 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 02:27:29.445975 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:29.445979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:29.446043 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:29.446177 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:29.447396 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.447949 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.449220 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.449456 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.449577 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.452516 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:29.452523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:29.452529 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:29.452565 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:29.454348 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.455027 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.455030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.455031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.455135 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.455573 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.455575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.455576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.455771 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:29.455893 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:29.456421 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 02:27:29.456581 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:27:29.457458 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.444012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:55359)
0807 02:27:29.444730 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:29.444840 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:29.445740 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:29.446359 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:29.446367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:29.446756 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:29.447237 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.447251 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.448091 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.448095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.449081 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.449093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.449653 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.449662 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.449828 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.450502 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:29.450520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:29.452334 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:29.454487 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.454516 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.454532 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.454908 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.455262 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.455266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.455270 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.455464 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.455903 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:29.455908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:29.456161 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 02:27:29.456320 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 02:27:29.456557 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":104,"threads_started":1}
I20260807 02:27:29.458832 30492 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36375
I20260807 02:27:29.459131 30492 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:29.459794 30492 heartbeater.cc:511] Master 127.17.53.62:36375 requested a full tablet report, sending...
I20260807 02:27:29.461035 30006 ts_manager.cc:194] Registered new tserver with Master: 080dff63567e4ca3948829f38395a252 (127.17.53.2:40241)
I20260807 02:27:29.461615 30006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:55359
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:29.483669 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:29.600725 30499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:29.601025 30499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:29.601096 30499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:29.605132 30499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:29.605211 30499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:29.605302 30499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:29.610796 30499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:29.611892 30499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:29.613077 30499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:29.623821 30510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:29.623883 30499 server_base.cc:1034] running on GCE node
W20260807 02:27:29.623886 30509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:29.623812 30512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:29.624449 30499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:29.625240 30499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:29.626415 30499 hybrid_clock.cc:648] HybridClock initialized: now 1786069649626399 us; error 32 us; skew 500 ppm
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:29.629457 30499 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:29.630434 30499 webserver.cc:460] Webserver started at http://127.17.53.3:38369/ using document root <none> and password file <none>
I20260807 02:27:29.631006 30499 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:29.631067 30499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:29.631264 30499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:29.633467 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "74254f0f05bf475bbc6210f4d9021c50"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "1dcff16dfc88f26eef5554fc2ea615b1"
server_key_iv: "2d4455e02f4a167b583057b38dda1154"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.634017 30499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "74254f0f05bf475bbc6210f4d9021c50"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "1dcff16dfc88f26eef5554fc2ea615b1"
server_key_iv: "2d4455e02f4a167b583057b38dda1154"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.638283 30499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:29.641113 30520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.642549 30499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:27:29.642761 30499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "74254f0f05bf475bbc6210f4d9021c50"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "1dcff16dfc88f26eef5554fc2ea615b1"
server_key_iv: "2d4455e02f4a167b583057b38dda1154"
server_key_version: "encryptionkey@0"
I20260807 02:27:29.642879 30499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:29.660130 30499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:29.664170 30499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:29.664408 30499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:29.665153 30499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:29.666059 30499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:29.666107 30499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.666178 30499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:29.666220 30499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:29.675782 30499 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:39345
I20260807 02:27:29.675808 30693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:39345 every 8 connection(s)
I20260807 02:27:29.676738 30499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:29.681102 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 30499
I20260807 02:27:29.681255 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:29.681583 17620 external_mini_cluster.cc:1428] Setting key 37e5db47d6a2d844c57f7ed6048c3f9b
Aug 07 02:27:29 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069649, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:29.692129 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.678901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:38181 (local address 127.17.53.62:36375)
0807 02:27:29.679045 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:29.679049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:29.679933 (+ 884us) server_negotiation.cc:407] Connection header received
0807 02:27:29.681462 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:29.681465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:29.681529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:29.681667 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:29.683155 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.683697 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.685081 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.685314 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.685405 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.688168 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:29.688171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:29.688173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:29.688197 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:29.689808 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.690525 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.690529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.690531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.690619 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:29.691004 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:29.691008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:29.691010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:29.691212 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:29.691324 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:29.691834 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 02:27:29.691961 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:27:29.693051 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:29.679266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:38181)
0807 02:27:29.679755 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:29.679799 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:29.681181 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:29.681848 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:29.681859 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:29.682346 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:29.682992 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.683015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.683876 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.683879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.684941 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:29.684953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:29.685459 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:29.685462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:29.685611 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:29.686205 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:29.686239 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:29.687980 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:29.690092 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.690099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.690115 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.690409 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.690747 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:29.690750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:29.690752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:29.690902 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:29.691324 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:29.691333 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:29.691585 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 02:27:29.691791 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 02:27:29.692123 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260807 02:27:29.694268 30694 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36375
I20260807 02:27:29.694585 30694 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:29.695255 30694 heartbeater.cc:511] Master 127.17.53.62:36375 requested a full tablet report, sending...
I20260807 02:27:29.696442 30006 ts_manager.cc:194] Registered new tserver with Master: 74254f0f05bf475bbc6210f4d9021c50 (127.17.53.3:39345)
I20260807 02:27:29.697152 30006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:38181
I20260807 02:27:29.708420 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
Entry for principal oryx/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab.
WARNING: no policy specified for oryx/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
Entry for principal oryx/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab.
WARNING: no policy specified for oryx/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
Entry for principal oryx/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab.
I20260807 02:27:29.791359 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 30095
I20260807 02:27:29.808648 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 30291
I20260807 02:27:29.836143 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 30499
I20260807 02:27:29.859406 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 29959
I20260807 02:27:29.881188 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36375
--webserver_interface=127.17.53.62
--webserver_port=33121
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:29.995513 30708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:29.995766 30708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:29.995806 30708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:29.999758 30708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:29.999815 30708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:29.999835 30708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:29.999851 30708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:29.999866 30708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:30.005097 30708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=33121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:30.006162 30708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:30.007153 30708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:30.016069 30716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.016103 30715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.016113 30718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:30.016373 30708 server_base.cc:1034] running on GCE node
I20260807 02:27:30.017091 30708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:30.018263 30708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:30.019438 30708 hybrid_clock.cc:648] HybridClock initialized: now 1786069650019403 us; error 48 us; skew 500 ppm
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069650, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:30.022431 30708 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:30.023413 30708 webserver.cc:460] Webserver started at http://127.17.53.62:33121/ using document root <none> and password file <none>
I20260807 02:27:30.023995 30708 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:30.024113 30708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:30.028059 30708 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:30.030450 30726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.031718 30708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:30.031873 30708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "ff34027892d74eb792320fa246a7458a"
format_stamp: "Formatted at 2026-08-07 02:27:28 on dist-test-slave-g170"
server_key: "b0d4ac4315f36620e8a5f769847f6c80"
server_key_iv: "86e51d934f74389b35140585da37d423"
server_key_version: "encryptionkey@0"
I20260807 02:27:30.032423 30708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:30.042165 30708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:30.045066 30708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:30.045256 30708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:30.052627 30708 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36375
I20260807 02:27:30.052645 30809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36375 every 8 connection(s)
I20260807 02:27:30.053730 30708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:30.058073 30811 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:27:30.058143 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 30708
I20260807 02:27:30.058621 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:45961
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:36375
--webserver_port=36147
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:27:30.059864 30811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Bootstrap starting.
I20260807 02:27:30.077199 30811 log.cc:824] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:30.083843 30811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:27:30.084240 30811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Bootstrap complete.
I20260807 02:27:30.088603 30811 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:30.089108 30811 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34027892d74eb792320fa246a7458a, State: Initialized, Role: FOLLOWER
I20260807 02:27:30.089599 30811 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:30.089744 30811 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:30.089838 30811 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:30.089968 30811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:27:30.091856 30811 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:30.092188 30811 leader_election.cc:302] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff34027892d74eb792320fa246a7458a; no voters:
I20260807 02:27:30.092563 30811 leader_election.cc:288] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 02:27:30.092648 30821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 02:27:30.092917 30821 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [term 2 LEADER]: Becoming Leader. State: Replica: ff34027892d74eb792320fa246a7458a, State: Running, Role: LEADER
I20260807 02:27:30.093274 30821 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } }
I20260807 02:27:30.093845 30811 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:30.094277 30827 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff34027892d74eb792320fa246a7458a. Latest consensus state: current_term: 2 leader_uuid: "ff34027892d74eb792320fa246a7458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } } }
I20260807 02:27:30.094477 30827 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:30.094695 30826 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff34027892d74eb792320fa246a7458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34027892d74eb792320fa246a7458a" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36375 } } }
I20260807 02:27:30.094828 30826 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:30.095185 30831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:30.098658 30831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:30.099367 30831 catalog_manager.cc:1259] Loaded cluster ID: 893a77738d1542eba4f006c151b116d3
I20260807 02:27:30.099433 30831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:30.101800 30831 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:30.102464 30831 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff34027892d74eb792320fa246a7458a: Loaded TSK: 0
I20260807 02:27:30.103655 30831 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:27:30.182248 30819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:30.182547 30819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:30.182644 30819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:30.187091 30819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:30.187172 30819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:30.187260 30819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:30.192710 30819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.1.keytab
--principal=oryx/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:45961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=36147
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:30.193864 30819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:30.195019 30819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:30.206255 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.206272 30855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.206394 30858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:30.207221 30819 server_base.cc:1034] running on GCE node
I20260807 02:27:30.207665 30819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:30.208389 30819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:30.209601 30819 hybrid_clock.cc:648] HybridClock initialized: now 1786069650209548 us; error 72 us; skew 500 ppm
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069650, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:30.212738 30819 init.cc:377] Logged in from keytab as oryx/127.17.53.1@KRBTEST.COM (short username oryx)
I20260807 02:27:30.213711 30819 webserver.cc:460] Webserver started at http://127.17.53.1:36147/ using document root <none> and password file <none>
I20260807 02:27:30.214313 30819 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:30.214390 30819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:30.218513 30819 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:30.220808 30868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.222374 30819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:30.222570 30819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "a5d3c535fbfa4f53be710b192620f1b2"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "8208219ef470153407eca81db056872a"
server_key_iv: "df0017e576d5906de16ea2a588f1cfcf"
server_key_version: "encryptionkey@0"
I20260807 02:27:30.223379 30819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:30.246654 30819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:30.250162 30819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:30.250385 30819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:30.251020 30819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:30.251920 30819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:30.251993 30819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.252069 30819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:30.252121 30819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.261523 30819 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45961
I20260807 02:27:30.261555 31028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45961 every 8 connection(s)
I20260807 02:27:30.262545 30819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:30.266767 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 30819
I20260807 02:27:30.267356 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:40241
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:36375
--webserver_port=33097
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.279322 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.264946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:46659)
0807 02:27:30.265696 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.265751 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.266672 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.268728 (+ 2056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.268740 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.269314 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.269977 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.269996 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.271356 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.271362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.272994 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.273010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.274355 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.274360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.274574 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.275291 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.275310 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.278529 (+ 3219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":151,"threads_started":1}
W20260807 02:27:30.279881 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.264857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:46659 (local address 127.17.53.62:36375)
0807 02:27:30.265339 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.265356 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.265907 (+ 551us) server_negotiation.cc:407] Connection header received
0807 02:27:30.267357 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.267374 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.267829 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.268192 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.270209 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.271209 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.273143 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.273935 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.274114 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.278886 (+ 4772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:46659: BlockingRecv error: recv got EOF from 127.17.53.1:46659 (error 108)
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
W20260807 02:27:30.280057 31029 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36375 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.53.62:36375: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.285912 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.280564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:58881)
0807 02:27:30.280852 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.280864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.280975 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.281242 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.281245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.281417 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.281510 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.281515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.282257 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.282263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.282825 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.282833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.283313 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.283316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.283409 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.283932 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.283945 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.285747 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":126,"threads_started":1}
W20260807 02:27:30.285984 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.280633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:58881 (local address 127.17.53.62:36375)
0807 02:27:30.280750 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.280754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.280930 (+ 176us) server_negotiation.cc:407] Connection header received
0807 02:27:30.281080 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.281083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.281132 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.281228 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.281629 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.282142 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.282951 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.283187 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.283304 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.285875 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:58881: BlockingRecv error: recv got EOF from 127.17.53.1:58881 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.291355 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.286495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:41461)
0807 02:27:30.286591 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.286607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.286682 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.286908 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.286910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.287060 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.287130 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.287135 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.287851 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.287856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.288369 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.288376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.288892 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.288896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.288983 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.289472 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.289484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.291224 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 02:27:30.291570 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.286587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:41461 (local address 127.17.53.62:36375)
0807 02:27:30.286696 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.286701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.286718 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:30.286756 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.286759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.286798 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.286892 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.287242 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.287733 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.288490 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.288721 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.288889 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.291348 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:41461: BlockingRecv error: recv got EOF from 127.17.53.1:41461 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 02:27:30.291572 31029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:27:30.404611 31034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:30.404955 31034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:30.405045 31034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:30.409076 31034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:30.409157 31034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:30.409252 31034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:30.414541 31034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.2.keytab
--principal=oryx/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:40241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=33097
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:30.415624 31034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:30.416684 31034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:30.426608 31045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.426607 31049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.426630 31046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:30.426947 31034 server_base.cc:1034] running on GCE node
I20260807 02:27:30.427373 31034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:30.428045 31034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:30.429195 31034 hybrid_clock.cc:648] HybridClock initialized: now 1786069650429175 us; error 29 us; skew 500 ppm
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069650, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:30.432838 31034 init.cc:377] Logged in from keytab as oryx/127.17.53.2@KRBTEST.COM (short username oryx)
I20260807 02:27:30.433779 31034 webserver.cc:460] Webserver started at http://127.17.53.2:33097/ using document root <none> and password file <none>
I20260807 02:27:30.434372 31034 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:30.434484 31034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:30.438513 31034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:27:30.440694 31060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.442076 31034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:30.442246 31034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "080dff63567e4ca3948829f38395a252"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "54880d0fe5ce649f4639fe3996e578a3"
server_key_iv: "165be267e20019ab477253cef5778b77"
server_key_version: "encryptionkey@0"
I20260807 02:27:30.442859 31034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:30.470786 31034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:30.474092 31034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:30.474321 31034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:30.475006 31034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:30.475907 31034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:30.475984 31034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.476058 31034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:30.476102 31034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.486285 31034 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40241
I20260807 02:27:30.486323 31242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40241 every 8 connection(s)
I20260807 02:27:30.487383 31034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:30.487797 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 31034
I20260807 02:27:30.488375 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:39345
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:36375
--webserver_port=38369
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 02:27:30.501230 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.489508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:47303 (local address 127.17.53.62:36375)
0807 02:27:30.489685 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.489691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.490922 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 02:27:30.491862 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.491870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.491950 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.492047 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.493455 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.494520 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.496165 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.496660 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.496849 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.501027 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:47303: BlockingRecv error: recv got EOF from 127.17.53.2:47303 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:27:30.501725 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.489935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:47303)
0807 02:27:30.490725 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.490782 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.491631 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.492275 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.492282 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.492652 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.493271 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.493288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.494774 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.494781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.496026 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.496038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.496906 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.496909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.497062 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.497829 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.497848 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.500742 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":160,"threads_started":1}
W20260807 02:27:30.502465 31243 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36375 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.53.62:36375: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.509086 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.503041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:49709)
0807 02:27:30.503316 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.503329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.503399 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.503679 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.503681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.503826 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.503927 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.503931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.505103 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.505106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.505643 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.505651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.506268 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.506271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.506341 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.506888 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.506899 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.508976 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208}
W20260807 02:27:30.509270 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.503191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:49709 (local address 127.17.53.62:36375)
0807 02:27:30.503358 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.503365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.503386 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:30.503481 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.503486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.503552 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.503655 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.504053 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.504993 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.505760 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.506144 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.506248 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.509145 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:49709: BlockingRecv error: recv got EOF from 127.17.53.2:49709 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.515420 31246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.509805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:40697)
0807 02:27:30.509934 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.509952 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.510036 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.510338 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.510340 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.510479 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.510558 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.510563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.511642 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.511645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.512184 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.512193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.512822 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.512828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.512897 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.513457 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.513473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.515330 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 02:27:30.515604 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.510000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:40697 (local address 127.17.53.62:36375)
0807 02:27:30.510106 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.510110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.510123 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:27:30.510161 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.510176 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.510218 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.510307 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.510671 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.511508 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.512307 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.512650 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.512757 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.515491 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:40697: BlockingRecv error: recv got EOF from 127.17.53.2:40697 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 02:27:30.515823 31243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:27:30.613946 31245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:30.614229 31245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:30.614276 31245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:30.618494 31245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:30.618553 31245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:30.618643 31245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:30.624348 31245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/krb5kdc/oryx_127.17.53.3.keytab
--principal=oryx/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:39345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=38369
--tserver_master_addrs=127.17.53.62:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:30.625504 31245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:30.626634 31245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:30.636889 31258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.636896 31255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:30.636896 31254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:30.637317 31245 server_base.cc:1034] running on GCE node
I20260807 02:27:30.637693 31245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:30.638331 31245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:30.639541 31245 hybrid_clock.cc:648] HybridClock initialized: now 1786069650639501 us; error 54 us; skew 500 ppm
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069650, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:30.642704 31245 init.cc:377] Logged in from keytab as oryx/127.17.53.3@KRBTEST.COM (short username oryx)
I20260807 02:27:30.643544 31245 webserver.cc:460] Webserver started at http://127.17.53.3:38369/ using document root <none> and password file <none>
I20260807 02:27:30.644119 31245 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:30.644193 31245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:30.648358 31245 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:30.650707 31267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.652037 31245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:30.652166 31245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "74254f0f05bf475bbc6210f4d9021c50"
format_stamp: "Formatted at 2026-08-07 02:27:29 on dist-test-slave-g170"
server_key: "1dcff16dfc88f26eef5554fc2ea615b1"
server_key_iv: "2d4455e02f4a167b583057b38dda1154"
server_key_version: "encryptionkey@0"
I20260807 02:27:30.652741 31245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:30.668337 31245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:30.671942 31245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:30.672164 31245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:30.672875 31245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:30.673766 31245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:30.673815 31245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.673856 31245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:30.673905 31245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:30.683878 31245 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:39345
I20260807 02:27:30.683902 31435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:39345 every 8 connection(s)
I20260807 02:27:30.684937 31245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:30.687484 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 31245
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 02:27:30.697012 31031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.686868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:54145 (local address 127.17.53.62:36375)
0807 02:27:30.687011 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.687015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.688239 (+ 1224us) server_negotiation.cc:407] Connection header received
0807 02:27:30.689422 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.689426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.689472 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.689555 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.691038 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.691524 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.692653 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.692923 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.693033 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.696901 (+ 3868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:54145: BlockingRecv error: recv got EOF from 127.17.53.3:54145 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:27:30.697788 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.687218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:54145)
0807 02:27:30.688043 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.688088 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.689201 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.689759 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.689765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.690248 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.690901 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.690918 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.691651 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.691654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.692523 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.692533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.693080 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.693084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.693267 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.693882 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.693920 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.696739 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":122,"threads_started":1}
I20260807 02:27:30.698206 31440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.688620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.17.53.62:36375)
0807 02:27:30.688907 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.688913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.688928 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:30.688986 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.688989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.689044 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.689141 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.689914 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.690710 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.691622 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.691856 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.691988 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.693588 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:30.693594 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:30.693607 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:30.693634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:30.695784 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:30.696213 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:30.696220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:30.696228 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:30.696400 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:30.696672 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:30.696678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:30.696680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:30.697139 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:30.697326 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:30.697755 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 02:27:30.698004 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
W20260807 02:27:30.698343 31436 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36375 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.53.62:36375: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.704136 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.698815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:50411)
0807 02:27:30.699053 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.699067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.699152 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.699393 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.699396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.699550 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.699638 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.699643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.700397 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.700402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.701279 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.701293 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.701783 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.701787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.701882 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.702366 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.702387 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.704047 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167}
W20260807 02:27:30.704288 31440 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.698925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50411 (local address 127.17.53.62:36375)
0807 02:27:30.699020 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.699023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.699111 (+ 88us) server_negotiation.cc:407] Connection header received
0807 02:27:30.699239 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.699241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.699287 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.699349 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.699753 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.700283 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.701416 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.701648 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.701772 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.704177 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:50411: BlockingRecv error: recv got EOF from 127.17.53.3:50411 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:30 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:30.709314 31438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:30.704739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:47793)
0807 02:27:30.704881 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:30.704897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:30.704982 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:30.705218 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:30.705220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:30.705347 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:30.705417 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.705422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.706131 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.706134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.706635 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:30.706675 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.707089 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.707092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:30.707150 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.707544 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:30.707556 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:30.709227 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 02:27:30.709443 31440 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:30.704851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:47793 (local address 127.17.53.62:36375)
0807 02:27:30.705001 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:30.705004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:30.705016 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:27:30.705045 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:30.705047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:30.705078 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:30.705141 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:30.705528 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.706006 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.706801 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:30.706996 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:30.707078 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:30.709337 (+ 2259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:47793: BlockingRecv error: recv got EOF from 127.17.53.3:47793 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 02:27:30.709538 31436 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:31.298656 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:31.292185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:35865)
0807 02:27:31.292530 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:31.292546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:31.292650 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:31.292979 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:31.292982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:31.293211 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:31.293353 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.293360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.294282 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.294291 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.295203 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.295216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.295737 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.295741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.295895 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.296421 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:31.296433 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:31.298474 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":77,"threads_started":1}
W20260807 02:27:31.298735 31444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:31.292283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35865 (local address 127.17.53.62:36375)
0807 02:27:31.292600 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:31.292603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:31.292617 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:31.292739 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:31.292741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:31.292808 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:31.292886 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:31.293485 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.294157 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.295331 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.295559 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.295705 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.298630 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:35865: BlockingRecv error: recv got EOF from 127.17.53.1:35865 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":107,"threads_started":1}
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:31.522523 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:31.516512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:45047)
0807 02:27:31.516833 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:31.516848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:31.516961 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:31.517224 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:31.517226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:31.517422 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:31.517523 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.517529 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.518345 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.518347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.518927 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.518936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.519408 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.519410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.519473 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.520041 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:31.520053 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:31.522318 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":109,"threads_started":1}
W20260807 02:27:31.522603 31444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:31.516572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:45047 (local address 127.17.53.62:36375)
0807 02:27:31.516819 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:31.516825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:31.516928 (+ 103us) server_negotiation.cc:407] Connection header received
0807 02:27:31.517036 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:31.517039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:31.517088 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:31.517194 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:31.517636 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.518238 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.519053 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.519297 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.519390 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.522467 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:45047: BlockingRecv error: recv got EOF from 127.17.53.2:45047 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:31 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:31.715821 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:31.710220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:33979)
0807 02:27:31.710487 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:31.710507 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:31.710608 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:31.710817 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:31.710819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:31.711007 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:31.711128 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.711133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.712006 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.712010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.712606 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:31.712616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.713092 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.713095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:31.713171 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.713685 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:31.713696 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:31.715677 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
W20260807 02:27:31.716069 31444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:31.710281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:33979 (local address 127.17.53.62:36375)
0807 02:27:31.710420 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:31.710424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:31.710574 (+ 150us) server_negotiation.cc:407] Connection header received
0807 02:27:31.710682 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:31.710684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:31.710731 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:31.710810 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:31.711275 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.711880 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.712738 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:31.712987 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:31.713072 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:31.715927 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:33979: BlockingRecv error: recv got EOF from 127.17.53.3:33979 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:32.305575 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:32.299621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:38167)
0807 02:27:32.299981 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:32.299997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:32.300113 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:32.300400 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:32.300402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:32.300653 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:32.300841 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.300858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.301639 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.301643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.302358 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.302370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.302872 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.302876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.302970 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.303543 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:32.303555 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:32.305446 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":132,"threads_started":1}
W20260807 02:27:32.305721 31448 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:32.299705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:38167 (local address 127.17.53.62:36375)
0807 02:27:32.300014 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:32.300018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:32.300056 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:27:32.300207 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:32.300209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:32.300253 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:32.300350 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:32.300974 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.301515 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.302499 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.302745 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.302863 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.305585 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:38167: BlockingRecv error: recv got EOF from 127.17.53.1:38167 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:32.532280 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:32.523508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:59713)
0807 02:27:32.523842 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:32.523865 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:32.524007 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:32.524287 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:32.524290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:32.524517 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:32.524619 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.524625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.525427 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.525431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.526013 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.526022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.526528 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.526532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.526656 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.530023 (+ 3367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:32.530036 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:32.532071 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
W20260807 02:27:32.532349 31448 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:32.523555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:59713 (local address 127.17.53.62:36375)
0807 02:27:32.523714 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:32.523718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:32.523944 (+ 226us) server_negotiation.cc:407] Connection header received
0807 02:27:32.524109 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:32.524114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:32.524169 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:32.524256 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:32.524740 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.525307 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.526138 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.526401 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.526509 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.532211 (+ 5702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:59713: BlockingRecv error: recv got EOF from 127.17.53.2:59713 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:32 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:32.722968 31450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:32.716683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:33581)
0807 02:27:32.717093 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:32.717111 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:32.717279 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:32.717589 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:32.717592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:32.717811 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:32.717916 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.717922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.718719 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.718724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.719307 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:32.719315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.719797 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.719801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:32.719898 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.720474 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:32.720487 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:32.722824 (+ 2337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":176,"threads_started":1}
W20260807 02:27:32.723119 31448 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:32.716913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:33581 (local address 127.17.53.62:36375)
0807 02:27:32.717093 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:32.717098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:32.717210 (+ 112us) server_negotiation.cc:407] Connection header received
0807 02:27:32.717400 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:32.717405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:32.717464 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:32.717556 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:32.718043 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.718603 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.719435 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:32.719671 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:32.719790 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:32.722966 (+ 3176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:33581: BlockingRecv error: recv got EOF from 127.17.53.3:33581 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:33.318158 31451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:33.306436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:34361)
0807 02:27:33.306735 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:33.306748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:33.306840 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:33.307093 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:33.307095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:33.307288 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:33.307387 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.307392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.308299 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.308306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.308941 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.308952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.309479 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.309482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.309553 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.315777 (+ 6224us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:33.315794 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:33.318040 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":111,"threads_started":1}
W20260807 02:27:33.318471 31455 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:33.306542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:34361 (local address 127.17.53.62:36375)
0807 02:27:33.306784 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:33.306787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:33.306802 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:27:33.306927 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:33.306929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:33.306973 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:33.307053 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:33.307532 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.308133 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.309054 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.309325 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.309420 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.318247 (+ 8827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:34361: BlockingRecv error: recv got EOF from 127.17.53.1:34361 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":132,"threads_started":1}
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:33.541290 31456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:33.533291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:47261)
0807 02:27:33.533625 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:33.533641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:33.533797 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:33.534114 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:33.534117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:33.534355 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:33.534527 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.534535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.535881 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.535887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.536997 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.537014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.537920 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.537926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.538016 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.538593 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:33.538614 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:33.541082 (+ 2468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
W20260807 02:27:33.541328 31455 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:33.533724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:47261 (local address 127.17.53.62:36375)
0807 02:27:33.533891 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:33.533896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:33.533909 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:27:33.533956 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:33.533958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:33.533998 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:33.534087 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:33.534710 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.535713 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.537176 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.537614 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.537749 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.541209 (+ 3460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:47261: BlockingRecv error: recv got EOF from 127.17.53.2:47261 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:33 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:33.730607 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:33.723965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:43783)
0807 02:27:33.724306 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:33.724324 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:33.724478 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:33.724805 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:33.724809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:33.725022 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:33.725119 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.725125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.725938 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.725942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.726543 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:33.726552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.727066 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.727069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:33.727157 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.727719 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:33.727842 (+ 123us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:33.730481 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":95,"threads_started":1}
W20260807 02:27:33.730822 31455 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:33.724099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:43783 (local address 127.17.53.62:36375)
0807 02:27:33.724302 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:33.724307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:33.724412 (+ 105us) server_negotiation.cc:407] Connection header received
0807 02:27:33.724583 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:33.724587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:33.724646 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:33.724739 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:33.725241 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.725776 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.726717 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:33.726946 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:33.727060 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:33.730666 (+ 3606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:43783: BlockingRecv error: recv got EOF from 127.17.53.3:43783 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:34.324579 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:34.319041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:52235)
0807 02:27:34.319346 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:34.319361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:34.319450 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:34.319727 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:34.319729 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:34.319915 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:34.320020 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.320025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.320815 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.320818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.321399 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.321408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.321852 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.321855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.321934 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.322526 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:34.322538 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:34.324442 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
W20260807 02:27:34.324858 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:34.319132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:52235 (local address 127.17.53.62:36375)
0807 02:27:34.319413 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:34.319417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:34.319431 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:34.319538 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:34.319540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:34.319592 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:34.319672 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:34.320125 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.320651 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.321506 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.321727 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.321812 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.324617 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:52235: BlockingRecv error: recv got EOF from 127.17.53.1:52235 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":134,"threads_started":1}
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:34.548317 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:34.542202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:41153)
0807 02:27:34.542547 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:34.542560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:34.542696 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:34.543022 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:34.543025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:34.543242 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:34.543348 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.543354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.544113 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.544118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.544705 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.544713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.545236 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.545240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.545347 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.545867 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:34.545879 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:34.548189 (+ 2310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
W20260807 02:27:34.548452 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:34.542291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:41153 (local address 127.17.53.62:36375)
0807 02:27:34.542468 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:34.542473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:34.542638 (+ 165us) server_negotiation.cc:407] Connection header received
0807 02:27:34.542842 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:34.542845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:34.542895 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:34.542993 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:34.543470 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.543999 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.544887 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.545113 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.545223 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.548331 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:41153: BlockingRecv error: recv got EOF from 127.17.53.2:41153 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:34 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:34.737984 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:34.731450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:51043)
0807 02:27:34.731832 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:34.731846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:34.732001 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:34.732296 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:34.732300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:34.732518 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:34.732616 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.732622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.733456 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.733462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.734118 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:34.734130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.734609 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.734612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:34.734700 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.735294 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:34.735310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:34.737846 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":95,"threads_started":1}
W20260807 02:27:34.738149 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:34.731575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:51043 (local address 127.17.53.62:36375)
0807 02:27:34.731768 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:34.731773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:34.731939 (+ 166us) server_negotiation.cc:407] Connection header received
0807 02:27:34.732126 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:34.732130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:34.732178 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:34.732265 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:34.732751 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.733335 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.734254 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:34.734490 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:34.734602 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:34.737989 (+ 3387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:51043: BlockingRecv error: recv got EOF from 127.17.53.3:51043 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:35.331885 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:35.325572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:50817)
0807 02:27:35.325887 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:35.325901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:35.325998 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:35.326272 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:35.326275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:35.326482 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:35.326582 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.326588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.327421 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.327424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.328462 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.328472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.328979 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.328982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.329042 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.329537 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:35.329548 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:35.331764 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
W20260807 02:27:35.332003 31468 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:35.325637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:50817 (local address 127.17.53.62:36375)
0807 02:27:35.325910 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:35.325914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:35.325970 (+ 56us) server_negotiation.cc:407] Connection header received
0807 02:27:35.326087 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:35.326089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:35.326134 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:35.326215 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:35.326688 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.327312 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.328573 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.328881 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.328963 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.331873 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:50817: BlockingRecv error: recv got EOF from 127.17.53.1:50817 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:35.555220 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:35.549193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:45529)
0807 02:27:35.549538 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:35.549555 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:35.549727 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:35.550012 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:35.550015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:35.550242 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:35.550351 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.550357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.551129 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.551133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.551739 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.551748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.552248 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.552252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.552360 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.552956 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:35.552968 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:35.555028 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":110,"threads_started":1}
W20260807 02:27:35.555259 31468 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:35.549387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:45529 (local address 127.17.53.62:36375)
0807 02:27:35.549544 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:35.549551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:35.549616 (+ 65us) server_negotiation.cc:407] Connection header received
0807 02:27:35.549841 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:35.549844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:35.549901 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:35.549988 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:35.550479 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.551016 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.551874 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.552116 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.552232 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.555149 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:45529: BlockingRecv error: recv got EOF from 127.17.53.2:45529 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:35 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:35.744912 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:35.738886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:45473)
0807 02:27:35.739207 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:35.739220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:35.739360 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:35.739644 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:35.739647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:35.739867 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:35.739962 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.739968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.740741 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.740745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.741391 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:35.741401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.741877 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.741880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:35.741969 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.742477 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:35.742498 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:35.744663 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
W20260807 02:27:35.744912 31468 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:35.738967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:45473 (local address 127.17.53.62:36375)
0807 02:27:35.739148 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:35.739154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:35.739309 (+ 155us) server_negotiation.cc:407] Connection header received
0807 02:27:35.739463 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:35.739468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:35.739526 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:35.739614 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:35.740081 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.740622 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.741526 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:35.741761 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:35.741871 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:35.744815 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:45473: BlockingRecv error: recv got EOF from 127.17.53.3:45473 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:36.339069 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:36.332719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:40635)
0807 02:27:36.333075 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:36.333092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:36.333204 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:36.333473 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:36.333475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:36.333690 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:36.333885 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.333892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.334948 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.334951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.335635 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.335645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.336209 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.336213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.336325 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.336981 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:36.337002 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:36.338951 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":141,"threads_started":1}
W20260807 02:27:36.339227 31476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:36.332909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:40635 (local address 127.17.53.62:36375)
0807 02:27:36.333163 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:36.333167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:36.333181 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:36.333302 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:36.333305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:36.333350 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:36.333429 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:36.334019 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.334828 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.335820 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.336110 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.336191 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.339088 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:40635: BlockingRecv error: recv got EOF from 127.17.53.1:40635 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:36.561673 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:36.556110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:50865)
0807 02:27:36.556448 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:36.556461 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:36.556567 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:36.556902 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:36.556904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:36.557098 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:36.557198 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.557203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.558016 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.558019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.558604 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.558613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.559061 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.559064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.559126 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.559638 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:36.559649 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:36.561553 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":111,"threads_started":1}
W20260807 02:27:36.561843 31476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:36.556204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:50865 (local address 127.17.53.62:36375)
0807 02:27:36.556344 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:36.556369 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:36.556537 (+ 168us) server_negotiation.cc:407] Connection header received
0807 02:27:36.556698 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:36.556701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:36.556760 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:36.556859 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:36.557374 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.557912 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.558729 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.558960 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.559041 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.561705 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:50865: BlockingRecv error: recv got EOF from 127.17.53.2:50865 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:36 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:36.751523 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:36.745750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:37515)
0807 02:27:36.746095 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:36.746118 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:36.746276 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:36.746582 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:36.746585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:36.746817 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:36.746918 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.746924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.747683 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.747688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.748253 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:36.748270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.748732 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.748736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:36.748863 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.749415 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:36.749427 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:36.751399 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":92,"threads_started":1}
W20260807 02:27:36.751717 31476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:36.745881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:37515 (local address 127.17.53.62:36375)
0807 02:27:36.746059 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:36.746064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:36.746196 (+ 132us) server_negotiation.cc:407] Connection header received
0807 02:27:36.746404 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:36.746408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:36.746464 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:36.746553 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:36.747044 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.747569 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.748386 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:36.748612 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:36.748726 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:36.751571 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:37515: BlockingRecv error: recv got EOF from 127.17.53.3:37515 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:37.345808 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:37.339883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:41701)
0807 02:27:37.340217 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:37.340234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:37.340342 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:37.340587 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:37.340589 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:37.340846 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:37.340948 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.340954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.341841 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.341844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.342407 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.342416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.342984 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.342986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.343052 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.343735 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:37.343751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:37.345687 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":90,"threads_started":1}
W20260807 02:27:37.346091 31483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:37.339938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:41701 (local address 127.17.53.62:36375)
0807 02:27:37.340217 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:37.340221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:37.340312 (+ 91us) server_negotiation.cc:407] Connection header received
0807 02:27:37.340425 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:37.340427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:37.340472 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:37.340549 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:37.341075 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.341703 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.342533 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.342818 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.342976 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.345856 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:41701: BlockingRecv error: recv got EOF from 127.17.53.1:41701 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:37.574534 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:37.562561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:44637)
0807 02:27:37.562894 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:37.562907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:37.563016 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:37.563341 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:37.563343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:37.563540 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:37.563648 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.563653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.564470 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.564473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.565394 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.565407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.566047 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.566050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.566113 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.572400 (+ 6287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:37.572413 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:37.574406 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
W20260807 02:27:37.574864 31483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:37.562716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:44637 (local address 127.17.53.62:36375)
0807 02:27:37.562878 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:37.562882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:37.562983 (+ 101us) server_negotiation.cc:407] Connection header received
0807 02:27:37.563153 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:37.563156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:37.563208 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:37.563300 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:37.563757 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.564292 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.565559 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.565918 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.566023 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.574604 (+ 8581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:44637: BlockingRecv error: recv got EOF from 127.17.53.2:44637 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:37 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:37.758551 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:37.752544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:59305)
0807 02:27:37.752996 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:37.753012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:37.753169 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:37.753468 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:37.753471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:37.753697 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:37.753802 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.753808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.754682 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.754687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.755363 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:37.755373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.755857 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.755861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:37.755951 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.756472 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:37.756485 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:37.758432 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":122,"threads_started":1}
W20260807 02:27:37.758842 31483 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:37.752700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:59305 (local address 127.17.53.62:36375)
0807 02:27:37.752915 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:37.752931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:37.753101 (+ 170us) server_negotiation.cc:407] Connection header received
0807 02:27:37.753291 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:37.753299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:37.753350 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:37.753440 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:37.753931 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.754568 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.755492 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:37.755736 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:37.755852 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:37.758576 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:59305: BlockingRecv error: recv got EOF from 127.17.53.3:59305 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:38.353209 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:38.346671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:55455)
0807 02:27:38.346962 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:38.346976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:38.347093 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:38.347402 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:38.347406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:38.347648 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:38.347791 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.347801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.349159 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.349162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.349734 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.349743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.350223 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.350226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.350312 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.350881 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:38.350893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:38.353078 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
W20260807 02:27:38.353346 31488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:38.346874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:55455 (local address 127.17.53.62:36375)
0807 02:27:38.347134 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:38.347137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:38.347153 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:27:38.347189 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:38.347191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:38.347231 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:38.347336 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:38.347989 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.349028 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.349879 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.350112 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.350202 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.353210 (+ 3008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:55455: BlockingRecv error: recv got EOF from 127.17.53.1:55455 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:38.581833 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:38.575295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:55221)
0807 02:27:38.575657 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:38.575675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:38.575815 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:38.576166 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:38.576170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:38.576402 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:38.576507 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.576513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.577608 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.577614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.578254 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.578264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.578979 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.578983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.579078 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.579781 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:38.579796 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:38.581708 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":90,"threads_started":1}
W20260807 02:27:38.582055 31488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:38.575434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:55221 (local address 127.17.53.62:36375)
0807 02:27:38.575621 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:38.575702 (+ 81us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:38.575743 (+ 41us) server_negotiation.cc:407] Connection header received
0807 02:27:38.575938 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:38.575942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:38.576002 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:38.576119 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:38.576649 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.577475 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.578412 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.578788 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.578930 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.581905 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:55221: BlockingRecv error: recv got EOF from 127.17.53.2:55221 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:38 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:38.766474 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:38.759361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:57277)
0807 02:27:38.759684 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:38.759699 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:38.759803 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:38.760190 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:38.760193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:38.760393 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:38.760513 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.760520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.761894 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.761897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.762473 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:38.762483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.763218 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.763222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:38.763291 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.763948 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:38.763960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:38.766345 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":124,"threads_started":1}
W20260807 02:27:38.766744 31488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:38.759476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:57277 (local address 127.17.53.62:36375)
0807 02:27:38.759686 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:38.759693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:38.759761 (+ 68us) server_negotiation.cc:407] Connection header received
0807 02:27:38.759920 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:38.759925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:38.759999 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:38.760131 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:38.760662 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.761737 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.762614 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:38.763069 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:38.763228 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:38.766563 (+ 3335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:57277: BlockingRecv error: recv got EOF from 127.17.53.3:57277 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:39.360713 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:39.354256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:49375)
0807 02:27:39.354599 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:39.354612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:39.354771 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:39.355144 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:39.355147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:39.355366 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:39.355471 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.355477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.356459 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.356462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.357154 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.357165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.357756 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.357759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.357827 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.358357 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:39.358382 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:39.360517 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":77,"threads_started":1}
W20260807 02:27:39.360875 31495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:39.354463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49375 (local address 127.17.53.62:36375)
0807 02:27:39.354712 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:39.354717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:39.354735 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:27:39.354924 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:39.354927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:39.354977 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:39.355078 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:39.355626 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.356305 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.357327 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.357629 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.357730 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.360672 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:49375: BlockingRecv error: recv got EOF from 127.17.53.1:49375 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:39.588675 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:39.582760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:48407)
0807 02:27:39.583107 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:39.583120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:39.583283 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:39.583578 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:39.583581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:39.583842 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:39.583947 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.583954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.584727 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.584733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.585353 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.585362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.585838 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.585842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.585933 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.586458 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:39.586471 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:39.588487 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
W20260807 02:27:39.588716 31495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:39.582822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:48407 (local address 127.17.53.62:36375)
0807 02:27:39.583044 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:39.583049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:39.583208 (+ 159us) server_negotiation.cc:407] Connection header received
0807 02:27:39.583393 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:39.583396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:39.583450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:39.583544 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:39.584071 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.584607 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.585481 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.585715 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.585832 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.588613 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:48407: BlockingRecv error: recv got EOF from 127.17.53.2:48407 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:39 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:39.773653 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:39.767495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:50091)
0807 02:27:39.767920 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:39.767938 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:39.768075 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:39.768398 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:39.768401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:39.768601 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:39.768715 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.768721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.769608 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.769612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.770226 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:39.770236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.770751 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.770754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:39.770825 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.771461 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:39.771473 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:39.773532 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":160,"threads_started":1}
W20260807 02:27:39.773830 31495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:39.767650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50091 (local address 127.17.53.62:36375)
0807 02:27:39.767859 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:39.767865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:39.768050 (+ 185us) server_negotiation.cc:407] Connection header received
0807 02:27:39.768178 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:39.768183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:39.768244 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:39.768320 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:39.768920 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.769480 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.770390 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:39.770625 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:39.770724 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:39.773700 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:50091: BlockingRecv error: recv got EOF from 127.17.53.3:50091 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:40.367892 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:40.361715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:50243)
0807 02:27:40.362025 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:40.362040 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:40.362139 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:40.362427 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:40.362430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:40.362698 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:40.362812 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.362819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.363635 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.363638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.364281 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.364290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.364755 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.364866 (+ 111us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.364969 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.365479 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:40.365492 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:40.367718 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
W20260807 02:27:40.368225 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:40.361768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:50243 (local address 127.17.53.62:36375)
0807 02:27:40.362067 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:40.362071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:40.362090 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:27:40.362242 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:40.362244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:40.362288 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:40.362376 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:40.362940 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.363501 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.364390 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.364611 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.364687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.368014 (+ 3327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:50243: BlockingRecv error: recv got EOF from 127.17.53.1:50243 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":126,"threads_started":1}
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:40.595795 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:40.589806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:34799)
0807 02:27:40.590102 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:40.590115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:40.590241 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:40.590532 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:40.590536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:40.590754 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:40.590863 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.590869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.591641 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.591647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.592231 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.592240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.592715 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.592719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.592860 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.593377 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:40.593389 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:40.595639 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
W20260807 02:27:40.595891 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:40.589870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:34799 (local address 127.17.53.62:36375)
0807 02:27:40.590034 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:40.590039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:40.590194 (+ 155us) server_negotiation.cc:407] Connection header received
0807 02:27:40.590352 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:40.590356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:40.590405 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:40.590503 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:40.590987 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.591525 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.592356 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.592598 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.592710 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.595768 (+ 3058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:34799: BlockingRecv error: recv got EOF from 127.17.53.2:34799 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:40 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:40.781008 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:40.774899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:33633)
0807 02:27:40.775314 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:40.775328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:40.775432 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:40.775762 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:40.775765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:40.776012 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:40.776136 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.776143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.777055 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.777059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.777733 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:40.777743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.778189 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.778192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:40.778256 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.778749 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:40.778760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:40.780864 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":129,"threads_started":1}
W20260807 02:27:40.781097 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:40.775122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:33633 (local address 127.17.53.62:36375)
0807 02:27:40.775312 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:40.775317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:40.775414 (+ 97us) server_negotiation.cc:407] Connection header received
0807 02:27:40.775529 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:40.775532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:40.775589 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:40.775682 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:40.776226 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.776956 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.777850 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:40.778066 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:40.778144 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:40.781003 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:33633: BlockingRecv error: recv got EOF from 127.17.53.3:33633 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:41.375321 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:41.369153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:34969)
0807 02:27:41.369524 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:41.369544 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:41.369649 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:41.370013 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:41.370016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:41.370265 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:41.370414 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.370423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.371451 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.371455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.372031 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.372039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.372534 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.372537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.372626 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.373169 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:41.373180 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:41.375199 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
W20260807 02:27:41.375490 31510 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:41.369270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:34969 (local address 127.17.53.62:36375)
0807 02:27:41.369582 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:41.369586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:41.369604 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:27:41.369791 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:41.369794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:41.369843 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:41.369940 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:41.370566 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.371325 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.372164 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.372409 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.372524 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.375348 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:34969: BlockingRecv error: recv got EOF from 127.17.53.1:34969 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":121,"threads_started":1}
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:41.602362 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:41.596612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:54453)
0807 02:27:41.596934 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:41.596951 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:41.597113 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:41.597430 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:41.597434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:41.597655 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:41.597766 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.597772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.598565 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.598570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.599142 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.599151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.599629 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.599632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.599726 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.600260 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:41.600272 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:41.602223 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":94,"threads_started":1}
W20260807 02:27:41.602540 31510 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:41.596685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:54453 (local address 127.17.53.62:36375)
0807 02:27:41.596919 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:41.596924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:41.597049 (+ 125us) server_negotiation.cc:407] Connection header received
0807 02:27:41.597247 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:41.597251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:41.597303 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:41.597395 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:41.597893 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.598446 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.599277 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.599509 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.599622 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.602402 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:54453: BlockingRecv error: recv got EOF from 127.17.53.2:54453 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:41 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:41.787644 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:41.781975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:46813)
0807 02:27:41.782314 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:41.782328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:41.782435 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:41.782724 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:41.782727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:41.782917 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:41.783026 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.783032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.783816 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.783819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.784384 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:41.784392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.784911 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.784914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:41.784983 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.785591 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:41.785613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:41.787524 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
W20260807 02:27:41.787915 31510 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:41.782048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:46813 (local address 127.17.53.62:36375)
0807 02:27:41.782201 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:41.782206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:41.782387 (+ 181us) server_negotiation.cc:407] Connection header received
0807 02:27:41.782513 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:41.782515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:41.782562 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:41.782631 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:41.783137 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.783688 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.784509 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:41.784724 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:41.784889 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:41.787665 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:46813: BlockingRecv error: recv got EOF from 127.17.53.3:46813 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:42.383010 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:42.376330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:55079)
0807 02:27:42.376662 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:42.376680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:42.376821 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:42.377196 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:42.377200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:42.377434 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:42.377541 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.377549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.378624 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.378629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.379205 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.379214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.379820 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.379825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.379918 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.380495 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:42.380509 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:42.382820 (+ 2311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":82,"threads_started":1}
W20260807 02:27:42.383090 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:42.376391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:55079 (local address 127.17.53.62:36375)
0807 02:27:42.376725 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:42.376729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:42.376746 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:42.376944 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:42.376948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:42.377014 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:42.377171 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:42.377686 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.378481 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.379339 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.379685 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.379816 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.382953 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:55079: BlockingRecv error: recv got EOF from 127.17.53.1:55079 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":68,"threads_started":1}
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:42.615360 31516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:42.603192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:37777)
0807 02:27:42.603557 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:42.603576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:42.603751 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:42.604058 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:42.604062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:42.604297 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:42.604414 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.604421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.605331 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.605338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.605941 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.605951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.606458 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.606463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.606563 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.613089 (+ 6526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:42.613104 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:42.615230 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
W20260807 02:27:42.615554 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:42.603373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:37777 (local address 127.17.53.62:36375)
0807 02:27:42.603559 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:42.603565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:42.603677 (+ 112us) server_negotiation.cc:407] Connection header received
0807 02:27:42.603862 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:42.603867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:42.603929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:42.604027 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:42.604551 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.605202 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.606083 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.606328 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.606453 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.615386 (+ 8933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:37777: BlockingRecv error: recv got EOF from 127.17.53.2:37777 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:42 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:42.794059 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:42.788425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:59545)
0807 02:27:42.788726 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:42.788740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:42.788891 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:42.789191 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:42.789193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:42.789384 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:42.789481 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.789486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.790289 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.790292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.790850 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:42.790858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.791352 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.791355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:42.791421 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.791949 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:42.791960 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:42.793904 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
W20260807 02:27:42.794260 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:42.788607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:59545 (local address 127.17.53.62:36375)
0807 02:27:42.788820 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:42.788824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:42.788841 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:27:42.789020 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:42.789023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:42.789074 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:42.789162 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:42.789596 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.790172 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.790969 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:42.791208 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:42.791317 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:42.794112 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:59545: BlockingRecv error: recv got EOF from 127.17.53.3:59545 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:43.389561 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:43.383813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:46427)
0807 02:27:43.384123 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:43.384137 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:43.384233 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:43.384497 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:43.384499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:43.384689 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:43.384889 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.384897 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.385799 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.385801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.386357 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.386365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.386820 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.386822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.386886 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.387466 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:43.387479 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:43.389443 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
W20260807 02:27:43.389788 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:43.383911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:46427 (local address 127.17.53.62:36375)
0807 02:27:43.384170 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:43.384174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:43.384188 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:43.384320 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:43.384322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:43.384368 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:43.384450 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:43.385056 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.385676 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.386466 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.386715 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.386800 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.389627 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:46427: BlockingRecv error: recv got EOF from 127.17.53.1:46427 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:43.621833 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:43.616234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:41851)
0807 02:27:43.616601 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:43.616620 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:43.616746 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:43.617123 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:43.617126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:43.617320 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:43.617421 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.617426 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.618126 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.618128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.618698 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.618706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.619125 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.619128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.619192 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.619698 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:43.619711 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:43.621650 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":86,"threads_started":1}
W20260807 02:27:43.621893 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:43.616368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:41851 (local address 127.17.53.62:36375)
0807 02:27:43.616514 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:43.616519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:43.616699 (+ 180us) server_negotiation.cc:407] Connection header received
0807 02:27:43.616869 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:43.616872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:43.616932 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:43.617021 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:43.617523 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.618030 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.618811 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.619025 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.619104 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.621769 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:41851: BlockingRecv error: recv got EOF from 127.17.53.2:41851 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:43 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:43.801015 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:43.794955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:54673)
0807 02:27:43.795231 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:43.795244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:43.795386 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:43.795670 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:43.795673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:43.795904 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:43.796000 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.796005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.796861 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.796865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.797461 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:43.797470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.797969 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.797973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:43.798064 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.798591 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:43.798604 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:43.800841 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
W20260807 02:27:43.801110 31520 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:43.794983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:54673 (local address 127.17.53.62:36375)
0807 02:27:43.795136 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:43.795140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:43.795323 (+ 183us) server_negotiation.cc:407] Connection header received
0807 02:27:43.795491 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:43.795494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:43.795545 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:43.795641 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:43.796129 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.796687 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.797588 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:43.797832 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:43.797947 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:43.800975 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:54673: BlockingRecv error: recv got EOF from 127.17.53.3:54673 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:44.397351 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:44.390794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:40821)
0807 02:27:44.391155 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:44.391176 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:44.391302 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:44.391717 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:44.391721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:44.391974 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:44.392082 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.392089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.392985 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.392991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.393630 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.393641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.394155 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.394161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.394279 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.394870 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:44.394881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:44.397167 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":91,"threads_started":1}
W20260807 02:27:44.397488 31524 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:44.390871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:40821 (local address 127.17.53.62:36375)
0807 02:27:44.391328 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:44.391333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:44.391354 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:44.391408 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:44.391412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:44.391468 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:44.391560 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:44.392229 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.392860 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.393753 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.393996 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.394129 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.397322 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:40821: BlockingRecv error: recv got EOF from 127.17.53.1:40821 (error 108)
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":86,"threads_started":1}
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:44.628708 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:44.622688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:41635)
0807 02:27:44.623035 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:44.623051 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:44.623209 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:44.623515 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:44.623518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:44.623780 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:44.623883 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.623889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.624684 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.624687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.625347 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.625357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.625840 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.625844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.625930 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.626502 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:44.626515 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:44.628568 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
W20260807 02:27:44.628918 31524 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:44.622865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:41635 (local address 127.17.53.62:36375)
0807 02:27:44.623039 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:44.623046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:44.623141 (+ 95us) server_negotiation.cc:407] Connection header received
0807 02:27:44.623321 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:44.623324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:44.623376 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:44.623464 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:44.624006 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.624561 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.625485 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.625721 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.625834 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.628720 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:41635: BlockingRecv error: recv got EOF from 127.17.53.2:41635 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:44 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.3@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:44.808086 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:44.801869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.3:49329)
0807 02:27:44.802163 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:44.802177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:44.802297 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:44.802728 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:44.802731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:44.802954 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:44.803088 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.803096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.803916 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.803920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.804535 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:44.804545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.805116 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.805120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:44.805234 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.805872 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:44.805895 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:44.807937 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
W20260807 02:27:44.808293 31524 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:44.801946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:49329 (local address 127.17.53.62:36375)
0807 02:27:44.802101 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:44.802105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:44.802252 (+ 147us) server_negotiation.cc:407] Connection header received
0807 02:27:44.802424 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:44.802427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:44.802507 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:44.802588 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:44.803241 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.803789 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.804666 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:44.804974 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:44.805053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:44.808138 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.3:49329: BlockingRecv error: recv got EOF from 127.17.53.3:49329 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 02:27:45 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:45 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.1@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:45.404006 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:45.398394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.1:43407)
0807 02:27:45.398687 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:45.398702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:45.398801 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:45.399060 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:45.399062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:45.399261 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:45.399364 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:45.399370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.400129 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.400133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:45.400708 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:45.400717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.401241 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.401245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:45.401311 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:45.401811 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:45.401823 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:45.403873 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
W20260807 02:27:45.404170 31531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:45.398462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:43407 (local address 127.17.53.62:36375)
0807 02:27:45.398744 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:45.398747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:45.398761 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:27:45.398896 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:45.398898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:45.398942 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:45.399019 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:45.399471 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.400019 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.400857 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.401098 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.401181 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:45.404029 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.1:43407: BlockingRecv error: recv got EOF from 127.17.53.1:43407 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
Aug 07 02:27:45 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 02:27:45 dist-test-slave-g170 krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.53.2@KRBTEST.COM for oryx/127.17.53.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 02:27:45.635521 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:45.629670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36375 (local address 127.17.53.2:60505)
0807 02:27:45.630112 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:45.630126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:45.630269 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:45.630569 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:45.630573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:45.630790 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:45.630898 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:45.630904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.631689 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.631694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:45.632287 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:45.632296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.632794 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.632806 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:45.632904 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:45.633412 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:45.633424 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:45.635331 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.53.62:36375: Server oryx/127.17.53.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":97,"threads_started":1}
W20260807 02:27:45.635578 31531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:45.629905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60505 (local address 127.17.53.62:36375)
0807 02:27:45.630085 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:45.630090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:45.630221 (+ 131us) server_negotiation.cc:407] Connection header received
0807 02:27:45.630387 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:45.630391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:45.630446 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:45.630535 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:45.631024 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.631571 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.632415 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.632645 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.632753 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:45.635463 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.53.2:60505: BlockingRecv error: recv got EOF from 127.17.53.2:60505 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:27:45.688094 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 30819
I20260807 02:27:45.707984 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 31034
I20260807 02:27:45.722057 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 31245
I20260807 02:27:45.731259 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 30708
2026-08-07T02:27:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18386 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T02:27:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:45Z Disabled control of system clock
I20260807 02:27:45.774138 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41183
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 02:27:45.902720 31537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:45.902995 31537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:45.903030 31537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:45.907159 31537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:45.907231 31537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:45.907248 31537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:45.907264 31537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:45.907281 31537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:45.912073 31537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:45.913215 31537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:45.914597 31537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:45.923332 31553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:45.923418 31548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:45.923417 31549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:45.923588 31537 server_base.cc:1034] running on GCE node
I20260807 02:27:45.924232 31537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:45.925478 31537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:45.926687 31537 hybrid_clock.cc:648] HybridClock initialized: now 1786069665926658 us; error 40 us; skew 500 ppm
I20260807 02:27:45.928581 31537 webserver.cc:460] Webserver started at http://127.17.53.62:41967/ using document root <none> and password file <none>
I20260807 02:27:45.929292 31537 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:45.929405 31537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:45.929661 31537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:45.931855 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "2e33d7bb3091478fa318bb1e0d10fe3c"
format_stamp: "Formatted at 2026-08-07 02:27:45 on dist-test-slave-g170"
server_key: "e9aa59b3619003fb2a2d5b40f33aa730"
server_key_iv: "94b4f1027a51087d69ba2e6e9e58003a"
server_key_version: "encryptionkey@0"
I20260807 02:27:45.932447 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "2e33d7bb3091478fa318bb1e0d10fe3c"
format_stamp: "Formatted at 2026-08-07 02:27:45 on dist-test-slave-g170"
server_key: "e9aa59b3619003fb2a2d5b40f33aa730"
server_key_iv: "94b4f1027a51087d69ba2e6e9e58003a"
server_key_version: "encryptionkey@0"
I20260807 02:27:45.936240 31537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:27:45.938962 31566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:45.940517 31537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:27:45.940713 31537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "2e33d7bb3091478fa318bb1e0d10fe3c"
format_stamp: "Formatted at 2026-08-07 02:27:45 on dist-test-slave-g170"
server_key: "e9aa59b3619003fb2a2d5b40f33aa730"
server_key_iv: "94b4f1027a51087d69ba2e6e9e58003a"
server_key_version: "encryptionkey@0"
I20260807 02:27:45.940913 31537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:45.973698 31537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:45.974576 31537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:45.974826 31537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:45.983037 31537 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:41183
I20260807 02:27:45.983063 31651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:41183 every 8 connection(s)
I20260807 02:27:45.984127 31537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:45.987661 31653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:45.991405 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 31537
I20260807 02:27:45.991505 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:45.991834 17620 external_mini_cluster.cc:1428] Setting key c38073994bba29d10007716ad9108d1a
I20260807 02:27:45.993675 31653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: Bootstrap starting.
I20260807 02:27:45.996186 31653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:45.997082 31653 log.cc:824] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:45.999462 31653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: No bootstrap required, opened a new log
I20260807 02:27:46.002398 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:45.993343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.17.53.62:41183)
0807 02:27:45.994026 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:45.994043 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:45.994079 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:27:45.994768 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:45.994786 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:45.995200 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:45.995494 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:45.995905 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.997158 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.998323 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:45.998883 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:45.999031 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.000111 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:46.000117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:46.000132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:46.000587 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:46.000649 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:46.000672 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:46.000913 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:46.001073 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 02:27:46.001416 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":142,"threads_started":1}
I20260807 02:27:46.003063 31653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } }
I20260807 02:27:46.003347 31653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:46.003409 31653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e33d7bb3091478fa318bb1e0d10fe3c, State: Initialized, Role: FOLLOWER
I20260807 02:27:46.003904 31653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } }
I20260807 02:27:46.004066 31653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:46.004160 31653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:46.004264 31653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:46.005419 31653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } }
I20260807 02:27:46.005765 31653 leader_election.cc:302] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e33d7bb3091478fa318bb1e0d10fe3c; no voters:
I20260807 02:27:46.006093 31653 leader_election.cc:288] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:46.006291 31659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:46.006670 31659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [term 1 LEADER]: Becoming Leader. State: Replica: 2e33d7bb3091478fa318bb1e0d10fe3c, State: Running, Role: LEADER
I20260807 02:27:46.007061 31659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } }
I20260807 02:27:46.007519 31653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:46.008989 31660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } } }
I20260807 02:27:46.009133 31660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:46.009507 31663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e33d7bb3091478fa318bb1e0d10fe3c. Latest consensus state: current_term: 1 leader_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e33d7bb3091478fa318bb1e0d10fe3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41183 } } }
I20260807 02:27:46.009604 31663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c [sys.catalog]: This master's current role is: LEADER
W20260807 02:27:46.012451 31681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:46.012603 31681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:46.012739 31675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:46.013473 31675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:46.019161 31675 catalog_manager.cc:1347] Generated new cluster ID: a71006d5c91e4a21a78a2d11c2893151
I20260807 02:27:46.019232 31675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:46.038918 31675 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:46.040160 31675 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:46.046972 31675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e33d7bb3091478fa318bb1e0d10fe3c: Generated new TSK 0
I20260807 02:27:46.047639 31675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:46.059562 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--builtin_ntp_servers=127.17.53.20:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 02:27:46.192019 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:46.192273 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:46.192312 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:46.196332 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:46.196388 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:46.196440 31690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:46.201171 31690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:46.202304 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:46.203753 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:46.213913 31699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.213932 31702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.213980 31690 server_base.cc:1034] running on GCE node
W20260807 02:27:46.213932 31700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.215547 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:46.216202 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:46.217394 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786069666217367 us; error 43 us; skew 500 ppm
I20260807 02:27:46.219300 31690 webserver.cc:460] Webserver started at http://127.17.53.1:41369/ using document root <none> and password file <none>
I20260807 02:27:46.219923 31690 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:46.220016 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:46.220261 31690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:46.222406 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "baf6787ca1484c2ab231df8047e3c39f"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "ace623b9f15f8c933443d561280c803a"
server_key_iv: "0b086c5c0d91afdde5c555a4246b58b6"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.222967 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "baf6787ca1484c2ab231df8047e3c39f"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "ace623b9f15f8c933443d561280c803a"
server_key_iv: "0b086c5c0d91afdde5c555a4246b58b6"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.226985 31690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:46.229729 31709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.231110 31690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 02:27:46.231304 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "baf6787ca1484c2ab231df8047e3c39f"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "ace623b9f15f8c933443d561280c803a"
server_key_iv: "0b086c5c0d91afdde5c555a4246b58b6"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.231442 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:46.247871 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:46.248575 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:46.248848 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:46.249455 31690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:46.250429 31690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:46.250502 31690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.250605 31690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:46.250653 31690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.262387 31690 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:34291
I20260807 02:27:46.262413 31885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:34291 every 8 connection(s)
I20260807 02:27:46.263342 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:46.269127 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 31690
I20260807 02:27:46.269254 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:46.269600 17620 external_mini_cluster.cc:1428] Setting key 86cc0993db75a6b91e69ff4b0226aa10
I20260807 02:27:46.272156 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--builtin_ntp_servers=127.17.53.20:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 02:27:46.275832 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.265285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:51575 (local address 127.17.53.62:41183)
0807 02:27:46.265439 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:46.265444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:46.266614 (+ 1170us) server_negotiation.cc:407] Connection header received
0807 02:27:46.268220 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:46.268223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:46.268290 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:46.268400 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:46.269632 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.270723 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.273226 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.273600 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.273707 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.275100 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:46.275104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:46.275107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:46.275121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:46.275148 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:46.275157 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:46.275242 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:46.275600 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 02:27:46.275674 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260807 02:27:46.277420 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.265774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41183 (local address 127.17.53.1:51575)
0807 02:27:46.266458 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:46.266499 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:46.267941 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:46.268580 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:46.268587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:46.268673 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:46.269442 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.269465 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.270920 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.270927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.273029 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.273045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.273928 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.273934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.274138 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.274806 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:46.274807 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:46.274850 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:46.274920 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:46.275245 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:46.275300 (+ 55us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:46.275312 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 02:27:46.275620 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 02:27:46.276125 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
I20260807 02:27:46.278955 31886 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41183
I20260807 02:27:46.279217 31886 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:46.279913 31886 heartbeater.cc:511] Master 127.17.53.62:41183 requested a full tablet report, sending...
I20260807 02:27:46.281924 31592 ts_manager.cc:194] Registered new tserver with Master: baf6787ca1484c2ab231df8047e3c39f (127.17.53.1:34291)
I20260807 02:27:46.283188 31592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.1:51575
W20260807 02:27:46.391368 31891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:46.391671 31891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:46.391745 31891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:46.395910 31891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:46.395996 31891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:46.396087 31891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:46.400837 31891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:46.401924 31891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:46.403323 31891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:46.413095 31900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.413266 31891 server_base.cc:1034] running on GCE node
W20260807 02:27:46.413115 31904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.413102 31901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.413859 31891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:46.414525 31891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:46.415731 31891 hybrid_clock.cc:648] HybridClock initialized: now 1786069666415715 us; error 28 us; skew 500 ppm
I20260807 02:27:46.417711 31891 webserver.cc:460] Webserver started at http://127.17.53.2:33261/ using document root <none> and password file <none>
I20260807 02:27:46.418347 31891 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:46.418463 31891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:46.418704 31891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:46.420809 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "0c184c44b580411cb852a0597abea6f1"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "09d6c1a35fb260dafbe2e41a1b10703c"
server_key_iv: "24311906046d1a95afb4292a458e6f0d"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.421422 31891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "0c184c44b580411cb852a0597abea6f1"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "09d6c1a35fb260dafbe2e41a1b10703c"
server_key_iv: "24311906046d1a95afb4292a458e6f0d"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.425295 31891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:27:46.428066 31916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.429438 31891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:46.429582 31891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "0c184c44b580411cb852a0597abea6f1"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "09d6c1a35fb260dafbe2e41a1b10703c"
server_key_iv: "24311906046d1a95afb4292a458e6f0d"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.429705 31891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:46.446137 31891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:46.446962 31891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:46.447228 31891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:46.447925 31891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:46.449129 31891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:46.449240 31891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.449343 31891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:46.449397 31891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.460577 31891 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:36943
I20260807 02:27:46.460598 32096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:36943 every 8 connection(s)
I20260807 02:27:46.461735 31891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:46.463249 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 31891
I20260807 02:27:46.463373 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:46.463658 17620 external_mini_cluster.cc:1428] Setting key 23fceb8975984af0d1c8ce30313a5a16
I20260807 02:27:46.466033 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--builtin_ntp_servers=127.17.53.20:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 02:27:46.472652 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.463995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:33883 (local address 127.17.53.62:41183)
0807 02:27:46.464162 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:46.464168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:46.465698 (+ 1530us) server_negotiation.cc:407] Connection header received
0807 02:27:46.467798 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:46.467801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:46.467849 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:46.467955 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:46.468965 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.469528 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.470675 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.470902 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.471015 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.472136 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:46.472140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:46.472143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:46.472163 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:46.472179 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:46.472187 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:46.472274 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:46.472439 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 02:27:46.472501 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:27:46.473555 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.464596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41183 (local address 127.17.53.2:33883)
0807 02:27:46.465530 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:46.465591 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:46.467552 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:46.468142 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:46.468149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:46.468234 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:46.468803 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.468828 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.469661 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.469665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.470543 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.470552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.471060 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.471064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.471233 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.471874 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:46.471875 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:46.471916 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:46.471979 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:46.472274 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:46.472286 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:46.472295 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:27:46.472436 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 02:27:46.472753 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":193,"threads_started":1}
I20260807 02:27:46.474717 32097 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41183
I20260807 02:27:46.474985 32097 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:46.475510 32097 heartbeater.cc:511] Master 127.17.53.62:41183 requested a full tablet report, sending...
I20260807 02:27:46.476630 31592 ts_manager.cc:194] Registered new tserver with Master: 0c184c44b580411cb852a0597abea6f1 (127.17.53.2:36943)
I20260807 02:27:46.477519 31592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.2:33883
W20260807 02:27:46.586213 32103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:46.586441 32103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:46.586476 32103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:46.590345 32103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:46.590389 32103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:46.590440 32103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:46.595024 32103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:46.595918 32103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:46.597173 32103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:46.606437 32116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.606422 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.606556 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.606603 32103 server_base.cc:1034] running on GCE node
I20260807 02:27:46.607247 32103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:46.608042 32103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:46.609242 32103 hybrid_clock.cc:648] HybridClock initialized: now 1786069666609234 us; error 46 us; skew 500 ppm
I20260807 02:27:46.611150 32103 webserver.cc:460] Webserver started at http://127.17.53.3:39859/ using document root <none> and password file <none>
I20260807 02:27:46.611702 32103 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:46.611804 32103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:46.612035 32103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:46.614049 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "315f50a66dfb4601b039b0c69bd475d9"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "e944dfe852d3e40a650dcae8f1506d0f"
server_key_iv: "dc146c73cc43b13fad7f6f3569f57908"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.614599 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "315f50a66dfb4601b039b0c69bd475d9"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "e944dfe852d3e40a650dcae8f1506d0f"
server_key_iv: "dc146c73cc43b13fad7f6f3569f57908"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.618561 32103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:27:46.620983 32126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.622185 32103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:46.622342 32103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "315f50a66dfb4601b039b0c69bd475d9"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "e944dfe852d3e40a650dcae8f1506d0f"
server_key_iv: "dc146c73cc43b13fad7f6f3569f57908"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.622457 32103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:46.655516 32103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:46.656287 32103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:46.656502 32103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:46.657173 32103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:46.658174 32103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:46.658250 32103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.658327 32103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:46.658393 32103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.667950 32103 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:41583
I20260807 02:27:46.667975 32298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:41583 every 8 connection(s)
I20260807 02:27:46.668973 32103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:46.673933 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 32103
I20260807 02:27:46.674082 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:46.674381 17620 external_mini_cluster.cc:1428] Setting key c36ef5c278f9ce204f27e0c2db7a4725
I20260807 02:27:46.679088 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.670957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:37281 (local address 127.17.53.62:41183)
0807 02:27:46.671125 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:46.671130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:46.672331 (+ 1201us) server_negotiation.cc:407] Connection header received
0807 02:27:46.673866 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:46.673869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:46.673918 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:46.673996 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:46.675126 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.675772 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.677306 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.677541 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.677626 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.678660 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:46.678664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:46.678667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:46.678680 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:46.678697 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:46.678706 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:46.678767 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:46.678931 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 02:27:46.678980 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 02:27:46.679929 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.671437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41183 (local address 127.17.53.3:37281)
0807 02:27:46.672159 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:46.672207 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:46.673595 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:46.674238 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:46.674248 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:46.674359 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:46.674964 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.674978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.675927 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.675930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.677188 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:46.677199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:46.677680 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:46.677683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:46.677828 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:46.678404 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:46.678406 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:46.678444 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:46.678505 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:46.678791 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:46.678800 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:46.678805 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 02:27:46.678909 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 02:27:46.679119 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
I20260807 02:27:46.681113 32299 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41183
I20260807 02:27:46.681385 32299 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:46.681952 32299 heartbeater.cc:511] Master 127.17.53.62:41183 requested a full tablet report, sending...
I20260807 02:27:46.682986 31592 ts_manager.cc:194] Registered new tserver with Master: 315f50a66dfb4601b039b0c69bd475d9 (127.17.53.3:41583)
I20260807 02:27:46.683564 31592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.3:37281
I20260807 02:27:46.688356 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 02:27:46.691386 31657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:46.690403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.17.53.62:41183)
0807 02:27:46.690531 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:46.690536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:46.690734 (+ 198us) server_negotiation.cc:407] Connection header received
0807 02:27:46.690894 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:46.690897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:46.690946 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:46.691031 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:46.691299 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38536: BlockingRecv error: recv got EOF from 127.0.0.1:38536 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:27:46.691706 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 31690
I20260807 02:27:46.705500 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 31891
I20260807 02:27:46.725884 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 32103
I20260807 02:27:46.736642 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 31537
2026-08-07T02:27:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (998 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T02:27:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:46Z Disabled control of system clock
I20260807 02:27:46.771865 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41849
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 02:27:46.891005 32311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:46.891275 32311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:46.891312 32311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:46.895191 32311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:46.895246 32311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:46.895262 32311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:46.895278 32311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:46.895293 32311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:46.900020 32311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41849
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:46.901068 32311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:46.902483 32311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:46.911182 32317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.911203 32320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:46.911230 32325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:46.911434 32311 server_base.cc:1034] running on GCE node
I20260807 02:27:46.911960 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:46.913101 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:46.914294 32311 hybrid_clock.cc:648] HybridClock initialized: now 1786069666914281 us; error 24 us; skew 500 ppm
I20260807 02:27:46.916081 32311 webserver.cc:460] Webserver started at http://127.17.53.62:37289/ using document root <none> and password file <none>
I20260807 02:27:46.916680 32311 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:46.916805 32311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:46.917028 32311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:46.919159 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "7235f690b6ad474b855e8d4712e2c460"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "7423afed401de944f8b5fc5ac59aca32"
server_key_iv: "3ebba3b3ff4e9b143a63f1f64e921f04"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.919749 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "7235f690b6ad474b855e8d4712e2c460"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "7423afed401de944f8b5fc5ac59aca32"
server_key_iv: "3ebba3b3ff4e9b143a63f1f64e921f04"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.923604 32311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 02:27:46.926398 32338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:46.927639 32311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:46.927865 32311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "7235f690b6ad474b855e8d4712e2c460"
format_stamp: "Formatted at 2026-08-07 02:27:46 on dist-test-slave-g170"
server_key: "7423afed401de944f8b5fc5ac59aca32"
server_key_iv: "3ebba3b3ff4e9b143a63f1f64e921f04"
server_key_version: "encryptionkey@0"
I20260807 02:27:46.928036 32311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:46.933249 32311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:46.933915 32311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:46.934120 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:46.942116 32311 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:41849
I20260807 02:27:46.942191 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:41849 every 8 connection(s)
I20260807 02:27:46.943195 32311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:46.946589 32424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:46.947906 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 32311
I20260807 02:27:46.948071 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:46.948386 17620 external_mini_cluster.cc:1428] Setting key 5e0985c76a37c36ed29fd670efb0e018
I20260807 02:27:46.953249 32424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460: Bootstrap starting.
I20260807 02:27:46.954393 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:46.949947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.17.53.62:41849)
0807 02:27:46.950622 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:46.950648 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:46.950686 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:27:46.951411 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:46.951428 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:46.951798 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:46.952091 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:46.953005 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:46.953010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:46.953023 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:46.953034 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:46.953094 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:46.953113 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:46.953236 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:46.953382 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 02:27:46.953660 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":146,"threads_started":1}
I20260807 02:27:46.955489 32424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:46.956223 32424 log.cc:824] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:46.958161 32424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460: No bootstrap required, opened a new log
I20260807 02:27:46.961701 32424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } }
I20260807 02:27:46.961944 32424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:46.962004 32424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7235f690b6ad474b855e8d4712e2c460, State: Initialized, Role: FOLLOWER
I20260807 02:27:46.962553 32424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } }
I20260807 02:27:46.962725 32424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:46.962796 32424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:46.962915 32424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:46.964116 32424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } }
I20260807 02:27:46.964507 32424 leader_election.cc:302] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7235f690b6ad474b855e8d4712e2c460; no voters:
I20260807 02:27:46.964865 32424 leader_election.cc:288] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:46.965063 32434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:46.965322 32434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [term 1 LEADER]: Becoming Leader. State: Replica: 7235f690b6ad474b855e8d4712e2c460, State: Running, Role: LEADER
I20260807 02:27:46.965633 32434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } }
I20260807 02:27:46.966235 32424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:46.967252 32437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7235f690b6ad474b855e8d4712e2c460. Latest consensus state: current_term: 1 leader_uuid: "7235f690b6ad474b855e8d4712e2c460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } } }
I20260807 02:27:46.967384 32437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:46.967723 32436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7235f690b6ad474b855e8d4712e2c460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7235f690b6ad474b855e8d4712e2c460" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41849 } } }
I20260807 02:27:46.967825 32436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:46.968120 32449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:46.971052 32449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:46.976938 32449 catalog_manager.cc:1347] Generated new cluster ID: eda16c1a656a4cdea78cecd347ecba1e
I20260807 02:27:46.977026 32449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:46.992412 32449 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:46.992898 32449 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:47.000619 32449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7235f690b6ad474b855e8d4712e2c460: Generated new TSK 0
I20260807 02:27:47.001477 32449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:27:47.004261 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--builtin_ntp_servers=127.17.53.20:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 02:27:47.123800 32465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:47.124085 32465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:47.124171 32465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:47.128171 32465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:47.128259 32465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:47.128353 32465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:47.133325 32465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:47.134472 32465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:47.135918 32465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:47.145733 32473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.145761 32480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.145749 32478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:47.146152 32465 server_base.cc:1034] running on GCE node
I20260807 02:27:47.146566 32465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:47.147238 32465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:47.148434 32465 hybrid_clock.cc:648] HybridClock initialized: now 1786069667148398 us; error 47 us; skew 500 ppm
I20260807 02:27:47.150396 32465 webserver.cc:460] Webserver started at http://127.17.53.1:45351/ using document root <none> and password file <none>
I20260807 02:27:47.151010 32465 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:47.151109 32465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:47.151363 32465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:47.153558 32465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "2bf0d6be327d446e893018385dcec1d6"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "0604a1ac50f6f862f42fb6b645a92347"
server_key_iv: "b063162eed7794ea38a7cead91c54d31"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.154147 32465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "2bf0d6be327d446e893018385dcec1d6"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "0604a1ac50f6f862f42fb6b645a92347"
server_key_iv: "b063162eed7794ea38a7cead91c54d31"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.157917 32465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260807 02:27:47.160697 32487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.162055 32465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:27:47.162174 32465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "2bf0d6be327d446e893018385dcec1d6"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "0604a1ac50f6f862f42fb6b645a92347"
server_key_iv: "b063162eed7794ea38a7cead91c54d31"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.162319 32465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:47.167075 32465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:47.167755 32465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:47.167939 32465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:47.168565 32465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:47.169523 32465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:47.169569 32465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.169633 32465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:47.169663 32465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.178462 32465 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:34769
I20260807 02:27:47.178493 32656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:34769 every 8 connection(s)
I20260807 02:27:47.179379 32465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:47.182240 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 32465
I20260807 02:27:47.182365 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:47.182711 17620 external_mini_cluster.cc:1428] Setting key 2c2e8b867adcd248de059c9c6f83096d
I20260807 02:27:47.184902 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--builtin_ntp_servers=127.17.53.20:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 02:27:47.186405 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.181349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:41109 (local address 127.17.53.62:41849)
0807 02:27:47.181520 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:47.181525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:47.182814 (+ 1289us) server_negotiation.cc:407] Connection header received
0807 02:27:47.184368 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:47.184371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:47.184411 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:47.184485 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:47.185786 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:47.185793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:47.185800 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:47.185820 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:47.185850 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:47.185865 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:47.185976 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:47.186145 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 02:27:47.186197 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 02:27:47.187227 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.181806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41849 (local address 127.17.53.1:41109)
0807 02:27:47.182652 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:47.182693 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:47.184160 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:47.184645 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:47.184652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:47.184737 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:47.185461 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:47.185462 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:47.185515 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:47.185602 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:47.185976 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:47.185985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:47.185991 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:27:47.186174 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 02:27:47.186383 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":164,"threads_started":1}
I20260807 02:27:47.188501 32657 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41849
I20260807 02:27:47.188743 32657 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:47.189174 32657 heartbeater.cc:511] Master 127.17.53.62:41849 requested a full tablet report, sending...
I20260807 02:27:47.190922 32372 ts_manager.cc:194] Registered new tserver with Master: 2bf0d6be327d446e893018385dcec1d6 (127.17.53.1:34769)
W20260807 02:27:47.308755 32661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:47.309088 32661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:47.309131 32661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:47.313017 32661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:47.313059 32661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:47.313112 32661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:47.317703 32661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:47.318703 32661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:47.320040 32661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:47.329725 32673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.329746 32670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.329767 32667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:47.330078 32661 server_base.cc:1034] running on GCE node
I20260807 02:27:47.330464 32661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:47.331120 32661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:47.332305 32661 hybrid_clock.cc:648] HybridClock initialized: now 1786069667332273 us; error 43 us; skew 500 ppm
I20260807 02:27:47.334188 32661 webserver.cc:460] Webserver started at http://127.17.53.2:43703/ using document root <none> and password file <none>
I20260807 02:27:47.334759 32661 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:47.334824 32661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:47.335083 32661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:47.337189 32661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "e29eaad17bed451db4d14378819498ae"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "ad61e11c690adc22744e8d4ecf040aa1"
server_key_iv: "8487b776b553d6d658718373d5c91c1e"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.337749 32661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "e29eaad17bed451db4d14378819498ae"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "ad61e11c690adc22744e8d4ecf040aa1"
server_key_iv: "8487b776b553d6d658718373d5c91c1e"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.341390 32661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:27:47.344106 32683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.345399 32661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260807 02:27:47.345537 32661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "e29eaad17bed451db4d14378819498ae"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "ad61e11c690adc22744e8d4ecf040aa1"
server_key_iv: "8487b776b553d6d658718373d5c91c1e"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.345640 32661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:47.350180 32661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:47.350778 32661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:47.350960 32661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:47.351503 32661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:47.352393 32661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:47.352442 32661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.352509 32661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:47.352550 32661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.361609 32661 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:37815
I20260807 02:27:47.361660 370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:37815 every 8 connection(s)
I20260807 02:27:47.362607 32661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:47.363039 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 32661
I20260807 02:27:47.363194 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:47.363557 17620 external_mini_cluster.cc:1428] Setting key 874bcb364320f6085e64a764e52e208b
I20260807 02:27:47.366209 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--builtin_ntp_servers=127.17.53.20:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 02:27:47.370723 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.364909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60853 (local address 127.17.53.62:41849)
0807 02:27:47.365071 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:47.365077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:47.366692 (+ 1615us) server_negotiation.cc:407] Connection header received
0807 02:27:47.368732 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:47.368736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:47.368810 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:47.368897 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:47.370226 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:47.370230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:47.370232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:47.370245 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:47.370273 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:47.370282 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:47.370371 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:47.370556 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 02:27:47.370600 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 02:27:47.372006 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.365513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41849 (local address 127.17.53.2:60853)
0807 02:27:47.366517 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:47.366576 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:47.368496 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:47.369100 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:47.369107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:47.369204 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:47.369883 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:47.369888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:47.369947 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:47.370042 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:47.370384 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:47.370397 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:47.370407 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 02:27:47.370593 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 02:27:47.370855 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":112,"threads_started":1}
I20260807 02:27:47.373314 372 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41849
I20260807 02:27:47.373602 372 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:47.374161 372 heartbeater.cc:511] Master 127.17.53.62:41849 requested a full tablet report, sending...
I20260807 02:27:47.375525 32372 ts_manager.cc:194] Registered new tserver with Master: e29eaad17bed451db4d14378819498ae (127.17.53.2:37815)
W20260807 02:27:47.487566 377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:47.488018 377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:47.488094 377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:47.492516 377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:47.492589 377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:47.492654 377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:47.498286 377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:47.499661 377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:47.501196 377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:47.511253 385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.511286 383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:47.511330 382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:47.511410 377 server_base.cc:1034] running on GCE node
I20260807 02:27:47.512214 377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:47.512916 377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:47.514137 377 hybrid_clock.cc:648] HybridClock initialized: now 1786069667514103 us; error 50 us; skew 500 ppm
I20260807 02:27:47.516027 377 webserver.cc:460] Webserver started at http://127.17.53.3:36947/ using document root <none> and password file <none>
I20260807 02:27:47.516605 377 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:47.516669 377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:47.516947 377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:47.519001 377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "b854aab793aa4000b67875801a52a4a5"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "4750bfd7d8ee9e22d6b219a154704b98"
server_key_iv: "a7b99c0dfb75de1e52aec37ae67cb3b1"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.519552 377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "b854aab793aa4000b67875801a52a4a5"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "4750bfd7d8ee9e22d6b219a154704b98"
server_key_iv: "a7b99c0dfb75de1e52aec37ae67cb3b1"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.523193 377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 02:27:47.525852 392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.527093 377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:47.527217 377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "b854aab793aa4000b67875801a52a4a5"
format_stamp: "Formatted at 2026-08-07 02:27:47 on dist-test-slave-g170"
server_key: "4750bfd7d8ee9e22d6b219a154704b98"
server_key_iv: "a7b99c0dfb75de1e52aec37ae67cb3b1"
server_key_version: "encryptionkey@0"
I20260807 02:27:47.527328 377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:47.532008 377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:47.532636 377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:47.532863 377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:47.533414 377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:47.534317 377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:47.534389 377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.534463 377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:47.534511 377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:47.543119 377 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:34645
I20260807 02:27:47.543139 563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:34645 every 8 connection(s)
I20260807 02:27:47.544003 377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:47.544883 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 377
I20260807 02:27:47.545002 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:47.545315 17620 external_mini_cluster.cc:1428] Setting key 6d7a95fdf2c4b408fc98338b7e5a61b2
I20260807 02:27:47.550352 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.546029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:58611 (local address 127.17.53.62:41849)
0807 02:27:47.546145 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:47.546156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:47.547356 (+ 1200us) server_negotiation.cc:407] Connection header received
0807 02:27:47.548745 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:47.548748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:47.548839 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:47.548906 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:47.549982 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:47.549985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:47.549987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:27:47.549997 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:47.550016 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:27:47.550024 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:47.550094 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:47.550207 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 02:27:47.550241 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 02:27:47.551250 567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:47.546502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41849 (local address 127.17.53.3:58611)
0807 02:27:47.547220 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:47.547271 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:47.548553 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:47.549034 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:47.549040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:47.549122 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:47.549742 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:27:47.549744 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:47.549783 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:27:47.549857 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:47.550113 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:47.550118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:47.550121 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 02:27:47.550245 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 02:27:47.550477 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260807 02:27:47.552336 564 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41849
I20260807 02:27:47.552533 564 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:47.552950 564 heartbeater.cc:511] Master 127.17.53.62:41849 requested a full tablet report, sending...
I20260807 02:27:47.553925 32372 ts_manager.cc:194] Registered new tserver with Master: b854aab793aa4000b67875801a52a4a5 (127.17.53.3:34645)
I20260807 02:27:47.559306 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 02:27:47.562125 32431 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:27:47.561372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.17.53.62:41849)
0807 02:27:47.561509 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:47.561512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:47.561686 (+ 174us) server_negotiation.cc:407] Connection header received
0807 02:27:47.561773 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:47.561776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:47.561809 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:47.561875 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:47.562034 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34328: BlockingRecv error: recv got EOF from 127.0.0.1:34328 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:27:47.562345 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 32465
I20260807 02:27:47.583992 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 32661
I20260807 02:27:47.597635 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 377
I20260807 02:27:47.615070 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 32311
2026-08-07T02:27:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (879 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:47 dist-test-slave-g170 krb5kdc[576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:47 dist-test-slave-g170 krb5kdc[576](info): set up 2 sockets
Aug 07 02:27:47 dist-test-slave-g170 krb5kdc[576](info): commencing operation
krb5kdc: starting...
W20260807 02:27:48.889580 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:48 dist-test-slave-g170 krb5kdc[576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:27:49.028055 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46145
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:46145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:49.152052 596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:49.152325 596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:49.152367 596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:49.156358 596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:27:49.156426 596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:49.156450 596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:49.156468 596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:27:49.156513 596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:27:49.161890 596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:46145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:49.162956 596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:49.163991 596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:49.173123 603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.173256 604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.173099 606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:49.173375 596 server_base.cc:1034] running on GCE node
I20260807 02:27:49.174044 596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:49.175153 596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:49.176323 596 hybrid_clock.cc:648] HybridClock initialized: now 1786069669176304 us; error 35 us; skew 500 ppm
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:49.179222 596 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:27:49.180042 596 webserver.cc:460] Webserver started at http://127.17.53.62:37527/ using document root <none> and password file <none>
I20260807 02:27:49.180583 596 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:49.180642 596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:49.180876 596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:49.182957 596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "354095a4de4a4ccca56dd641ac55caa0"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "e7f4d72eea72b276b002788e443e7e53"
server_key_iv: "88faeb9d0f9ef73b57b509ef0efe1aac"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.183508 596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "354095a4de4a4ccca56dd641ac55caa0"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "e7f4d72eea72b276b002788e443e7e53"
server_key_iv: "88faeb9d0f9ef73b57b509ef0efe1aac"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.187644 596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:27:49.190281 617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.191589 596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:27:49.191699 596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "354095a4de4a4ccca56dd641ac55caa0"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "e7f4d72eea72b276b002788e443e7e53"
server_key_iv: "88faeb9d0f9ef73b57b509ef0efe1aac"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.191789 596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:49.206440 596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:49.209707 596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:49.209865 596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:49.217569 596 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:46145
I20260807 02:27:49.217595 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:46145 every 8 connection(s)
I20260807 02:27:49.218564 596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:27:49.221475 690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:49.225733 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 596
I20260807 02:27:49.225878 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:27:49.226161 17620 external_mini_cluster.cc:1428] Setting key cddefd04c058985c9a2852a46e145479
I20260807 02:27:49.227586 690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: Bootstrap starting.
I20260807 02:27:49.230517 690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:49.231418 690 log.cc:824] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:49.233659 690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: No bootstrap required, opened a new log
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:49.238299 690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } }
I20260807 02:27:49.238606 690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:49.238669 690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354095a4de4a4ccca56dd641ac55caa0, State: Initialized, Role: FOLLOWER
I20260807 02:27:49.239192 690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } }
I20260807 02:27:49.239334 690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:27:49.239405 690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:27:49.239506 690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:49.240648 690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } }
I20260807 02:27:49.241055 690 leader_election.cc:302] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354095a4de4a4ccca56dd641ac55caa0; no voters:
I20260807 02:27:49.241407 690 leader_election.cc:288] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:27:49.241632 699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:49.241918 699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [term 1 LEADER]: Becoming Leader. State: Replica: 354095a4de4a4ccca56dd641ac55caa0, State: Running, Role: LEADER
I20260807 02:27:49.242288 699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } }
I20260807 02:27:49.242476 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:49.227803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.17.53.62:46145)
0807 02:27:49.228259 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:49.228279 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:49.228322 (+ 43us) server_negotiation.cc:407] Connection header received
0807 02:27:49.229230 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:49.229254 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:49.229672 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:49.230057 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:49.231005 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.232335 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.233362 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.234054 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.234232 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:49.236867 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:49.236874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:49.236891 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:49.236923 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:49.239006 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.239585 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.239593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.239599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.239749 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.240051 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.240054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.240056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.240592 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:49.240841 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:49.241191 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 02:27:49.241508 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":128,"threads_started":1}
I20260807 02:27:49.242794 690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:27:49.245082 700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354095a4de4a4ccca56dd641ac55caa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } } }
I20260807 02:27:49.245554 700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [sys.catalog]: This master's current role is: LEADER
I20260807 02:27:49.245738 701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354095a4de4a4ccca56dd641ac55caa0. Latest consensus state: current_term: 1 leader_uuid: "354095a4de4a4ccca56dd641ac55caa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354095a4de4a4ccca56dd641ac55caa0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46145 } } }
I20260807 02:27:49.245867 701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0 [sys.catalog]: This master's current role is: LEADER
W20260807 02:27:49.247720 717 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:27:49.247808 717 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:27:49.247888 720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:27:49.248603 720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:27:49.254956 720 catalog_manager.cc:1347] Generated new cluster ID: 4c922116e5ba4cb1a36cc53e1ab3ef7d
I20260807 02:27:49.255038 720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:27:49.275820 720 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:27:49.277156 720 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:27:49.283479 720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: Generated new TSK 0
I20260807 02:27:49.284214 720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:27:49.316988 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:49.449193 731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:49.449437 731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:49.449478 731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:49.453397 731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:49.453450 731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:49.453507 731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:27:49.458669 731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:49.459656 731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:49.460628 731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:49.470630 741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.470602 739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:49.470590 731 server_base.cc:1034] running on GCE node
W20260807 02:27:49.470592 738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:49.471220 731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:49.471905 731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:49.473106 731 hybrid_clock.cc:648] HybridClock initialized: now 1786069669473073 us; error 44 us; skew 500 ppm
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:49.476016 731 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:27:49.476995 731 webserver.cc:460] Webserver started at http://127.17.53.1:38547/ using document root <none> and password file <none>
I20260807 02:27:49.477594 731 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:49.477706 731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:49.477959 731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:49.480001 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "5904a702bff642a89d6838c207d46193"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "d54fc6db48086e0f7c90dedc86c1f51b"
server_key_iv: "efffc672db773f82f309a2dd41ad289f"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.480599 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "5904a702bff642a89d6838c207d46193"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "d54fc6db48086e0f7c90dedc86c1f51b"
server_key_iv: "efffc672db773f82f309a2dd41ad289f"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.484607 731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 02:27:49.488255 750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.489763 731 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:27:49.489928 731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "5904a702bff642a89d6838c207d46193"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "d54fc6db48086e0f7c90dedc86c1f51b"
server_key_iv: "efffc672db773f82f309a2dd41ad289f"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.490062 731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:49.519814 731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:49.523423 731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:49.523644 731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:49.524279 731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:49.525260 731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:49.525333 731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.525408 731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:49.525451 731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.534937 731 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:41551
I20260807 02:27:49.534970 918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:41551 every 8 connection(s)
I20260807 02:27:49.535859 731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:27:49.546046 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 731
I20260807 02:27:49.546177 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:27:49.546449 17620 external_mini_cluster.cc:1428] Setting key ff65ecf16222442556baf4f6acebdf31
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:49.551503 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:49.537853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:46277 (local address 127.17.53.62:46145)
0807 02:27:49.537994 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:49.537998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:49.538924 (+ 926us) server_negotiation.cc:407] Connection header received
0807 02:27:49.540375 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:49.540379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:49.540439 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:49.540551 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:49.541984 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.542873 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.544347 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.544576 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.544687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:49.547432 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:49.547435 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:49.547437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:49.547464 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:49.549185 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.549784 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.549790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.549792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.549886 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.550280 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.550286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.550287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.550483 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:49.550606 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:49.551147 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 02:27:49.551298 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:27:49.552356 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:49.538275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:46145 (local address 127.17.53.1:46277)
0807 02:27:49.538772 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:49.538815 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:49.540160 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:49.540717 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:49.540724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:49.541155 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:49.541830 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:49.541849 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.543025 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.543028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:49.544194 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:49.544206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.544718 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.544723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:49.544949 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:49.545479 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:49.545497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:49.547220 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:49.549346 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:49.549352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:49.549367 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:49.549656 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:49.549997 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:49.550001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:49.550003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:49.550164 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:49.550607 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:49.550615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:49.550889 (+ 274us) client_negotiation.cc:769] Sending connection context
0807 02:27:49.551100 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 02:27:49.551449 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260807 02:27:49.553742 921 heartbeater.cc:347] Connected to a master server at 127.17.53.62:46145
I20260807 02:27:49.554030 921 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:49.554688 921 heartbeater.cc:511] Master 127.17.53.62:46145 requested a full tablet report, sending...
I20260807 02:27:49.556574 636 ts_manager.cc:194] Registered new tserver with Master: 5904a702bff642a89d6838c207d46193 (127.17.53.1:41551)
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:27:49.558154 636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:46277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:27:49.575811 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:49.693698 928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:49.693966 928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:49.694010 928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:49.698199 928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:49.698263 928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:49.698323 928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:27:49.703676 928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:49.704725 928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:49.705982 928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:49.716126 935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.716113 939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.716132 934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:49.716544 928 server_base.cc:1034] running on GCE node
I20260807 02:27:49.717043 928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:49.717686 928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:49.718927 928 hybrid_clock.cc:648] HybridClock initialized: now 1786069669718913 us; error 35 us; skew 500 ppm
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:49.721959 928 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:27:49.722884 928 webserver.cc:460] Webserver started at http://127.17.53.2:46525/ using document root <none> and password file <none>
I20260807 02:27:49.723484 928 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:49.723582 928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:49.723829 928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:49.725993 928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "4def58650b5148abb755ad1a3321021f"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "95d785dcfec3c86e39a6fbe43090b0f6"
server_key_iv: "64561de4313c2ec07d3ee202b3f7a6f4"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.726598 928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "4def58650b5148abb755ad1a3321021f"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "95d785dcfec3c86e39a6fbe43090b0f6"
server_key_iv: "64561de4313c2ec07d3ee202b3f7a6f4"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.730665 928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 02:27:49.733605 949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.734982 928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:27:49.735138 928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "4def58650b5148abb755ad1a3321021f"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "95d785dcfec3c86e39a6fbe43090b0f6"
server_key_iv: "64561de4313c2ec07d3ee202b3f7a6f4"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.735265 928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:49.747076 928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:49.750554 928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:49.750788 928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:49.751425 928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:49.752329 928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:49.752405 928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.752468 928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:49.752494 928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.762253 928 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:41749
I20260807 02:27:49.762276 1134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:41749 every 8 connection(s)
I20260807 02:27:49.763273 928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:27:49.773021 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 928
I20260807 02:27:49.773135 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:27:49.773428 17620 external_mini_cluster.cc:1428] Setting key bffdaff6d4e9e244138cd1ce1aba9adc
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:49.778499 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:49.765317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:50991 (local address 127.17.53.62:46145)
0807 02:27:49.765494 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:49.765498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:49.766450 (+ 952us) server_negotiation.cc:407] Connection header received
0807 02:27:49.767843 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:49.767846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:49.767896 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:49.767976 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:49.769292 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.769837 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.771042 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.771309 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.771405 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:49.774352 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:49.774355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:49.774357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:49.774381 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:49.776157 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.776755 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.776788 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.776793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.776899 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:49.777309 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:49.777313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:49.777314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:49.777512 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:49.777629 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:49.778188 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 02:27:49.778320 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 02:27:49.779480 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:49.765710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:46145 (local address 127.17.53.2:50991)
0807 02:27:49.766260 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:49.766322 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:49.767620 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:49.768156 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:49.768162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:49.768539 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:49.769116 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:49.769131 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.769977 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.769981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:49.770896 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:49.770908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:49.771467 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:49.771470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:49.771615 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:49.772294 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:49.772315 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:49.774149 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:49.776309 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:49.776314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:49.776327 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:49.776619 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:49.777003 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:49.777006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:49.777007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:49.777191 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:49.777628 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:49.777635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:49.777927 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 02:27:49.778132 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 02:27:49.778491 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":125,"threads_started":1}
I20260807 02:27:49.780962 1135 heartbeater.cc:347] Connected to a master server at 127.17.53.62:46145
I20260807 02:27:49.781230 1135 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:49.781759 1135 heartbeater.cc:511] Master 127.17.53.62:46145 requested a full tablet report, sending...
I20260807 02:27:49.783061 636 ts_manager.cc:194] Registered new tserver with Master: 4def58650b5148abb755ad1a3321021f (127.17.53.2:41749)
I20260807 02:27:49.783982 636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:50991
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:27:49.807519 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:43279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:27:49.928707 1142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:27:49.929098 1142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:27:49.929148 1142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:27:49.933373 1142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:27:49.933451 1142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:27:49.933512 1142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:27:49.939231 1142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:43279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:46145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:27:49.940476 1142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:27:49.941604 1142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:27:49.952281 1148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.952283 1154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:27:49.952320 1150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:27:49.952593 1142 server_base.cc:1034] running on GCE node
I20260807 02:27:49.953248 1142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:27:49.953920 1142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:27:49.955138 1142 hybrid_clock.cc:648] HybridClock initialized: now 1786069669955105 us; error 61 us; skew 500 ppm
Aug 07 02:27:49 dist-test-slave-g170 krb5kdc[576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:27:49.958039 1142 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:27:49.958990 1142 webserver.cc:460] Webserver started at http://127.17.53.3:43945/ using document root <none> and password file <none>
I20260807 02:27:49.959539 1142 fs_manager.cc:359] Metadata directory not provided
I20260807 02:27:49.959599 1142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:27:49.959764 1142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:27:49.961890 1142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "4d16860193f64376870ac44f46540f48"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "81552a52bd986ab65c739faedb94e139"
server_key_iv: "0ab029ad91914193645f83b006c4750d"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.962512 1142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "4d16860193f64376870ac44f46540f48"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "81552a52bd986ab65c739faedb94e139"
server_key_iv: "0ab029ad91914193645f83b006c4750d"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.966516 1142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 02:27:49.969163 1165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.970510 1142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:27:49.970615 1142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "4d16860193f64376870ac44f46540f48"
format_stamp: "Formatted at 2026-08-07 02:27:49 on dist-test-slave-g170"
server_key: "81552a52bd986ab65c739faedb94e139"
server_key_iv: "0ab029ad91914193645f83b006c4750d"
server_key_version: "encryptionkey@0"
I20260807 02:27:49.970754 1142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:27:49.982206 1142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:27:49.985563 1142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:27:49.985777 1142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:27:49.986402 1142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:27:49.987287 1142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:27:49.987337 1142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.987378 1142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:27:49.987394 1142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:27:49.997360 1142 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:37251
I20260807 02:27:49.997393 1350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:37251 every 8 connection(s)
I20260807 02:27:49.998317 1142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:27:50.005964 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 1142
I20260807 02:27:50.006120 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:27:50.006534 17620 external_mini_cluster.cc:1428] Setting key ab7f007897b2409c7659b584f1becb13
Aug 07 02:27:50 dist-test-slave-g170 krb5kdc[576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069669, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:27:50.013873 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.000188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:44017 (local address 127.17.53.62:46145)
0807 02:27:50.000392 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.000397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.001315 (+ 918us) server_negotiation.cc:407] Connection header received
0807 02:27:50.002724 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.002727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.002777 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.002856 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:50.004031 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.004580 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.006469 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.006760 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.006862 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.009929 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:50.009933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:50.009935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:50.009965 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:50.011546 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:50.012112 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:50.012115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:50.012117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:50.012206 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:50.012601 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:50.012604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:50.012605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:50.012939 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:50.013034 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.013571 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.013736 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 02:27:50.014731 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.000609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:46145 (local address 127.17.53.3:44017)
0807 02:27:50.001143 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.001188 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.002501 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.002998 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.003005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.003380 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:27:50.003882 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.003895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.004962 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.004965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.006303 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.006319 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.006954 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.006959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.007155 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.007897 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:27:50.007916 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:27:50.009662 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:27:50.011671 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:50.011677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:50.011700 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:50.011998 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:50.012324 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:27:50.012326 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:27:50.012328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:27:50.012490 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:27:50.013045 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:27:50.013051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:27:50.013316 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.013482 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.013792 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":182,"threads_started":1}
I20260807 02:27:50.016083 1351 heartbeater.cc:347] Connected to a master server at 127.17.53.62:46145
I20260807 02:27:50.016371 1351 heartbeater.cc:464] Registering TS with master...
I20260807 02:27:50.017076 1351 heartbeater.cc:511] Master 127.17.53.62:46145 requested a full tablet report, sending...
I20260807 02:27:50.018277 636 ts_manager.cc:194] Registered new tserver with Master: 4d16860193f64376870ac44f46540f48 (127.17.53.3:37251)
I20260807 02:27:50.018929 636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:44017
I20260807 02:27:50.021322 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:27:50.031493 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.023370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.17.53.62:46145)
0807 02:27:50.023509 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.023517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.023724 (+ 207us) server_negotiation.cc:407] Connection header received
0807 02:27:50.023761 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.023764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.023805 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.023879 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:27:50.024620 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.025326 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.026171 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.026413 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.026512 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.028142 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:27:50.028145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:27:50.028148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:27:50.028174 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:27:50.029729 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:50.030224 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:50.030227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:50.030228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:50.030317 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:27:50.030620 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:27:50.030622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:27:50.030624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:27:50.030913 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:27:50.031029 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.031250 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.031358 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:27:50.034799 636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:27:50.038199 636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:27:50.053804 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.048475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:37251 (local address 127.0.0.1:51218)
0807 02:27:50.048782 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.048797 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.048886 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.049604 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.049607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.049627 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:50.049725 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.049734 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.050845 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.050849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.052285 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.052296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053295 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.053298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.053371 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.053387 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.053433 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.053596 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":47,"thread_start_us":105,"threads_started":1}
I20260807 02:27:50.053833 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.047816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41942 (local address 127.17.53.2:41749)
0807 02:27:50.048854 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.048871 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.048899 (+ 28us) server_negotiation.cc:407] Connection header received
0807 02:27:50.048982 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.048987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.049202 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.049379 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:50.049714 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.050920 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.052255 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053021 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.053332 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.053503 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.053561 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.053655 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":155,"threads_started":1}
I20260807 02:27:50.053969 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.047653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41551 (local address 127.0.0.1:46592)
0807 02:27:50.048062 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.048116 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.048242 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.049722 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.049725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.049744 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:50.049830 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.049836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.051384 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.051388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.052483 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.052491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053582 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.053585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.053689 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.053713 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.053768 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.053836 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
I20260807 02:27:50.054481 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.048660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.17.53.3:37251)
0807 02:27:50.049167 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.049178 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.049199 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:27:50.049279 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.049287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.049478 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.049624 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:50.049853 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.050644 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.052439 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053162 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.053996 (+ 834us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.054182 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.054239 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.054331 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":212,"threads_started":1}
I20260807 02:27:50.054576 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.048860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46592 (local address 127.17.53.1:41551)
0807 02:27:50.049205 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.049222 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.049252 (+ 30us) server_negotiation.cc:407] Connection header received
0807 02:27:50.049345 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.049351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.049550 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.049981 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:50.050140 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.051200 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.052605 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053436 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.054126 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.054292 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.054341 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.054423 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":166,"threads_started":1}
I20260807 02:27:50.053805 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.047653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:41749 (local address 127.0.0.1:41942)
0807 02:27:50.048429 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.048443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.048532 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.049388 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.049394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.049420 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:50.049558 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.049567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.051077 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.051081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.052110 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.052120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.053155 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.053159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.053294 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.053345 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.053445 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.053596 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":562,"thread_start_us":91,"threads_started":1}
I20260807 02:27:50.057346 1024 tablet_service.cc:1467] Processing CreateTablet for tablet c05fcafde03b4b2a9d706203abe14484 (DEFAULT_TABLE table=test-table [id=5dc1b25c8e7847c2bdf9f89c5649c080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:50.057490 1246 tablet_service.cc:1467] Processing CreateTablet for tablet c05fcafde03b4b2a9d706203abe14484 (DEFAULT_TABLE table=test-table [id=5dc1b25c8e7847c2bdf9f89c5649c080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:50.057482 824 tablet_service.cc:1467] Processing CreateTablet for tablet c05fcafde03b4b2a9d706203abe14484 (DEFAULT_TABLE table=test-table [id=5dc1b25c8e7847c2bdf9f89c5649c080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:27:50.058544 824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fcafde03b4b2a9d706203abe14484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:50.058548 1246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fcafde03b4b2a9d706203abe14484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:50.058545 1024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fcafde03b4b2a9d706203abe14484. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:27:50.064939 1373 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Bootstrap starting.
I20260807 02:27:50.066329 1375 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Bootstrap starting.
I20260807 02:27:50.066510 1374 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Bootstrap starting.
I20260807 02:27:50.067883 1373 tablet_bootstrap.cc:654] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:50.068317 1375 tablet_bootstrap.cc:654] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:50.068832 1374 tablet_bootstrap.cc:654] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Neither blocks nor log segments found. Creating new log.
I20260807 02:27:50.069003 1373 log.cc:824] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:50.069576 1375 log.cc:824] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:50.069674 1374 log.cc:824] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Log is configured to *not* fsync() on all Append() calls
I20260807 02:27:50.071115 1373 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: No bootstrap required, opened a new log
I20260807 02:27:50.071321 1373 ts_tablet_manager.cc:1397] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 02:27:50.071455 1374 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: No bootstrap required, opened a new log
I20260807 02:27:50.071575 1375 tablet_bootstrap.cc:492] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: No bootstrap required, opened a new log
I20260807 02:27:50.071659 1374 ts_tablet_manager.cc:1397] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:27:50.071758 1375 ts_tablet_manager.cc:1397] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 02:27:50.075369 1373 raft_consensus.cc:348] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.075398 1374 raft_consensus.cc:348] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.075503 1375 raft_consensus.cc:348] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.075649 1373 raft_consensus.cc:374] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:50.075661 1374 raft_consensus.cc:374] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:50.075718 1375 raft_consensus.cc:374] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:27:50.075716 1374 raft_consensus.cc:729] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5904a702bff642a89d6838c207d46193, State: Initialized, Role: FOLLOWER
I20260807 02:27:50.075712 1373 raft_consensus.cc:729] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4def58650b5148abb755ad1a3321021f, State: Initialized, Role: FOLLOWER
I20260807 02:27:50.075765 1375 raft_consensus.cc:729] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d16860193f64376870ac44f46540f48, State: Initialized, Role: FOLLOWER
I20260807 02:27:50.076184 1374 consensus_queue.cc:260] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.076195 1373 consensus_queue.cc:260] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.076201 1375 consensus_queue.cc:260] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.077229 1374 ts_tablet_manager.cc:1428] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:27:50.077230 1373 ts_tablet_manager.cc:1428] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 02:27:50.077275 1375 ts_tablet_manager.cc:1428] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 02:27:50.077299 921 heartbeater.cc:503] Master 127.17.53.62:46145 was elected leader, sending a full tablet report...
I20260807 02:27:50.077597 1135 heartbeater.cc:503] Master 127.17.53.62:46145 was elected leader, sending a full tablet report...
I20260807 02:27:50.077770 1351 heartbeater.cc:503] Master 127.17.53.62:46145 was elected leader, sending a full tablet report...
W20260807 02:27:50.249353 1352 tablet.cc:2369] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:27:50.264562 1136 tablet.cc:2369] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:50.275497 1382 raft_consensus.cc:482] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:27:50.275751 1382 raft_consensus.cc:504] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.279196 1382 leader_election.cc:288] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4def58650b5148abb755ad1a3321021f (127.17.53.2:41749), 5904a702bff642a89d6838c207d46193 (127.17.53.1:41551)
I20260807 02:27:50.282161 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.278431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:60259 (local address 127.17.53.1:41551)
0807 02:27:50.278564 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.278569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.278581 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:27:50.278630 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.278633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.278687 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.278753 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:50.279185 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.279803 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281068 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.281519 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281605 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.281689 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.281952 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.282027 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:27:50.282167 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.277696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41551 (local address 127.17.53.3:60259)
0807 02:27:50.278108 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.278128 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.278220 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.278852 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.278855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.278897 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:50.279058 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.279067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.279938 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.279941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.280934 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.280945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.281651 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.281806 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.281833 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.281907 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.281997 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":51,"spinlock_wait_cycles":6784}
I20260807 02:27:50.282173 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.278045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:34339 (local address 127.17.53.2:41749)
0807 02:27:50.278176 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.278181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.278434 (+ 253us) server_negotiation.cc:407] Connection header received
0807 02:27:50.278564 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.278568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.278628 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.278708 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:27:50.279449 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.279995 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281198 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.281614 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281719 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.281796 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.281991 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.282071 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 02:27:50.282192 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.277704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:41749 (local address 127.17.53.3:34339)
0807 02:27:50.278310 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.278325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:27:50.278452 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:27:50.278852 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:27:50.278855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:27:50.278896 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:27:50.279278 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.279285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.280124 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.280127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.281065 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:27:50.281077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.281740 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.281744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:27:50.281847 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.281866 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 02:27:50.281962 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 02:27:50.282027 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":310,"spinlock_wait_cycles":30464,"thread_start_us":134,"threads_started":1}
I20260807 02:27:50.282859 855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fcafde03b4b2a9d706203abe14484" candidate_uuid: "4d16860193f64376870ac44f46540f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5904a702bff642a89d6838c207d46193" is_pre_election: true
I20260807 02:27:50.282860 1059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fcafde03b4b2a9d706203abe14484" candidate_uuid: "4d16860193f64376870ac44f46540f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4def58650b5148abb755ad1a3321021f" is_pre_election: true
I20260807 02:27:50.283171 1059 raft_consensus.cc:2454] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d16860193f64376870ac44f46540f48 in term 0.
I20260807 02:27:50.283179 855 raft_consensus.cc:2454] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d16860193f64376870ac44f46540f48 in term 0.
I20260807 02:27:50.283675 1167 leader_election.cc:302] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d16860193f64376870ac44f46540f48, 5904a702bff642a89d6838c207d46193; no voters:
I20260807 02:27:50.284001 1382 raft_consensus.cc:2781] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:27:50.284099 1382 raft_consensus.cc:482] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:27:50.284204 1382 raft_consensus.cc:3037] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:50.285363 1382 raft_consensus.cc:504] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.285712 1382 leader_election.cc:288] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [CANDIDATE]: Term 1 election: Requested vote from peers 4def58650b5148abb755ad1a3321021f (127.17.53.2:41749), 5904a702bff642a89d6838c207d46193 (127.17.53.1:41551)
I20260807 02:27:50.286082 1059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fcafde03b4b2a9d706203abe14484" candidate_uuid: "4d16860193f64376870ac44f46540f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4def58650b5148abb755ad1a3321021f"
I20260807 02:27:50.286087 855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fcafde03b4b2a9d706203abe14484" candidate_uuid: "4d16860193f64376870ac44f46540f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5904a702bff642a89d6838c207d46193"
I20260807 02:27:50.286212 1059 raft_consensus.cc:3037] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:27:50.286228 855 raft_consensus.cc:3037] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 0 FOLLOWER]: Advancing to term 1
W20260807 02:27:50.287204 922 tablet.cc:2369] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:27:50.287233 1059 raft_consensus.cc:2454] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d16860193f64376870ac44f46540f48 in term 1.
I20260807 02:27:50.287395 855 raft_consensus.cc:2454] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d16860193f64376870ac44f46540f48 in term 1.
I20260807 02:27:50.287545 1171 leader_election.cc:302] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d16860193f64376870ac44f46540f48, 4def58650b5148abb755ad1a3321021f; no voters:
I20260807 02:27:50.287774 1382 raft_consensus.cc:2781] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:27:50.287989 1382 raft_consensus.cc:686] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 1 LEADER]: Becoming Leader. State: Replica: 4d16860193f64376870ac44f46540f48, State: Running, Role: LEADER
I20260807 02:27:50.288304 1382 consensus_queue.cc:237] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } }
I20260807 02:27:50.291523 637 catalog_manager.cc:5495] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d16860193f64376870ac44f46540f48 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "4d16860193f64376870ac44f46540f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d16860193f64376870ac44f46540f48" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 37251 } health_report { overall_health: HEALTHY } } }
I20260807 02:27:50.345114 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.341104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.17.53.3:37251)
0807 02:27:50.341256 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:27:50.341261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:27:50.341277 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:27:50.341407 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:27:50.341410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:27:50.341467 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:27:50.341584 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:27:50.341850 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.342380 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.343355 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:27:50.343620 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:27:50.343736 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:27:50.344096 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:27:50.344594 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:27:50.344735 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 02:27:50.344901 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 02:27:50.344962 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 02:27:50.351586 1059 raft_consensus.cc:1261] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 1 FOLLOWER]: Refusing update from remote peer 4d16860193f64376870ac44f46540f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:50.351586 855 raft_consensus.cc:1261] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193 [term 1 FOLLOWER]: Refusing update from remote peer 4d16860193f64376870ac44f46540f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:27:50.352444 1387 consensus_queue.cc:1035] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5904a702bff642a89d6838c207d46193" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:50.352773 1382 consensus_queue.cc:1035] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4def58650b5148abb755ad1a3321021f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:27:50.361636 1393 mvcc.cc:205] Tried to move back new op lower bound from 7315741369752248320 to 7315741369503567872. Current Snapshot: MvccSnapshot[applied={T|T < 7315741369752248320}]
I20260807 02:27:50.373171 637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:50.373351 637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:27:50.376295 637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 354095a4de4a4ccca56dd641ac55caa0: Sending DeleteTablet for 3 replicas of tablet c05fcafde03b4b2a9d706203abe14484
I20260807 02:27:50.377275 1024 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fcafde03b4b2a9d706203abe14484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:41942
I20260807 02:27:50.377517 824 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fcafde03b4b2a9d706203abe14484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46592
I20260807 02:27:50.378099 1412 tablet_replica.cc:330] T c05fcafde03b4b2a9d706203abe14484 P 5904a702bff642a89d6838c207d46193: stopping tablet replica
I20260807 02:27:50.378096 1246 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fcafde03b4b2a9d706203abe14484 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:27:50 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:51218
I20260807 02:27:50.378463 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 731
I20260807 02:27:50.393712 1414 tablet_replica.cc:330] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: stopping tablet replica
I20260807 02:27:50.394104 1414 raft_consensus.cc:2229] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:27:50.394579 1414 raft_consensus.cc:2258] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:50.395092 1411 tablet_replica.cc:330] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: stopping tablet replica
I20260807 02:27:50.395471 1411 raft_consensus.cc:2229] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:27:50.395850 1411 raft_consensus.cc:2258] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:27:50.397015 1414 ts_tablet_manager.cc:1905] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:27:50.397464 1411 ts_tablet_manager.cc:1905] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 02:27:50.399413 619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:41551 (error 108)
I20260807 02:27:50.400485 1411 ts_tablet_manager.cc:1918] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:50.400615 1411 log.cc:1196] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/c05fcafde03b4b2a9d706203abe14484
I20260807 02:27:50.400745 1414 ts_tablet_manager.cc:1918] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:27:50.401014 1411 ts_tablet_manager.cc:1939] T c05fcafde03b4b2a9d706203abe14484 P 4def58650b5148abb755ad1a3321021f: Deleting consensus metadata
I20260807 02:27:50.401228 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 928
I20260807 02:27:50.401317 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.400955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41551 (local address 127.0.0.1:46608)
0807 02:27:50.401109 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.401210 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 02:27:50.414834 619 catalog_manager.cc:4584] TS 5904a702bff642a89d6838c207d46193 (127.17.53.1:41551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c05fcafde03b4b2a9d706203abe14484: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41551: connect: Connection refused (error 111)
I20260807 02:27:50.400978 1414 log.cc:1196] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/c05fcafde03b4b2a9d706203abe14484
I20260807 02:27:50.423502 1414 ts_tablet_manager.cc:1939] T c05fcafde03b4b2a9d706203abe14484 P 4d16860193f64376870ac44f46540f48: Deleting consensus metadata
I20260807 02:27:50.425032 619 catalog_manager.cc:4841] TS 4d16860193f64376870ac44f46540f48 (127.17.53.3:37251): tablet c05fcafde03b4b2a9d706203abe14484 (table test-table [id=5dc1b25c8e7847c2bdf9f89c5649c080]) successfully deleted
I20260807 02:27:50.431563 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 1142
I20260807 02:27:50.432210 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:27:50.431844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:41749 (local address 127.0.0.1:41954)
0807 02:27:50.432050 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 02:27:50.432108 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.2:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 02:27:50.444623 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 596
2026-08-07T02:27:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2833 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:27:50 dist-test-slave-g170 krb5kdc[1419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:27:50 dist-test-slave-g170 krb5kdc[1419](info): set up 2 sockets
Aug 07 02:27:50 dist-test-slave-g170 krb5kdc[1419](info): commencing operation
krb5kdc: starting...
W20260807 02:27:51.709401 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:27:51 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:27:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:27:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangeradmin_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangeradmin_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangerlookup_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangerlookup_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
I20260807 02:27:51.896692 17620 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPRMQP4/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/postgres -l logfile start
2026-08-07 02:27:54.571 UTC [1456] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 02:27:54.571 UTC [1456] LOG: listening on IPv4 address "127.17.53.20", port 38239
2026-08-07 02:27:54.575 UTC [1456] LOG: listening on Unix socket "/tmp/.s.PGSQL.38239"
2026-08-07 02:27:54.598 UTC [1463] LOG: database system was shut down at 2026-08-07 02:27:53 UTC
2026-08-07 02:27:54.602 UTC [1456] LOG: database system is ready to accept connections
I20260807 02:27:55.778475 17620 mini_postgres.cc:96] Postgres bound to 38239
2026-08-07 02:27:55.788 UTC [1476] FATAL: database "slave" does not exist
127.17.53.20:38239 - accepting connections
I20260807 02:27:55.789175 17620 mini_ranger.cc:161] Starting Ranger...
I20260807 02:27:55.811566 17620 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 02:27:56.323016 17620 mini_ranger.cc:87] Created ranger Postgres database
I20260807 02:27:56.323123 17620 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin
2026-08-07 02:27:56,667 [I] DB FLAVOR :POSTGRES
2026-08-07 02:27:56,667 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 02:27:56,667 [I] Checking connection..
2026-08-07 02:27:56,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 02:27:56,970 [I] Checking connection passed.
2026-08-07 02:27:56,971 [I] --------- Verifying version history table ---------
2026-08-07 02:27:56,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:27:57,241 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 02:27:57,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:27:57,499 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 02:27:57,499 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 02:27:57,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 02:27:57.767 UTC [1573] WARNING: there is no transaction in progress
2026-08-07 02:27:57,777 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 02:27:57,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:27:58,040 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 02:27:58,041 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 02:27:58,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 02:27:58,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 02:27:58,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g170.c.gcp-upstream.internal','N') ;"
2026-08-07 02:27:58,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 02:27:59,035 [I] Table x_portal_user does not exist in database ranger
2026-08-07 02:27:59,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 02:27:59,296 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 02:27:59,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 02:27:59,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 02:27:59,835 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 02:27:59,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 02:28:00.832 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.850 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.869 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.885 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.901 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.920 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.977 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.983 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:00.999 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.011 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.023 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.039 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.050 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.057 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.064 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01.071 UTC [1731] WARNING: there is no transaction in progress
2026-08-07 02:28:01,748 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 02:28:01,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 02:28:02,024 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 02:28:02,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 02:28:02,288 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 02:28:02,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 02:28:02,560 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 02:28:02,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 02:28:02,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 02:28:03,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g170.c.gcp-upstream.internal';"
2026-08-07 02:28:03,339 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 02:28:03,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 02:28:03,607 [I] DB_PATCHES have already been applied
I20260807 02:28:03.613899 17620 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPRMQP4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 02:28:03.613966 17620 mini_ranger.cc:193] Using host: 127.17.53.20
I20260807 02:28:03.618753 17620 mini_ranger.cc:227] Ranger admin URL: http://127.17.53.20:38993
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 02:28:05.005060 17620 mini_ranger.cc:160] Time spent starting Ranger: real 9.216s user 0.000s sys 0.013s
Aug 07, 2026 2:28:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 2:28:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 2:28:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 2:28:26 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 2:28:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 02:28:27.705444 17620 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangerkms_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/rangerkms_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.53.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.53.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 02:28:27 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 02:28:27.801594 17620 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 02:28:27.813447 17620 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 02:28:28.761646 17620 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 02:28:28.761755 17620 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms
I20260807 02:28:28.761866 17620 mini_ranger_kms.cc:227] Using postgres at 127.17.53.20:38239
2026-08-07 02:28:29,141 [I] DB FLAVOR :POSTGRES
2026-08-07 02:28:29,141 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 02:28:29,142 [I] Checking connection
2026-08-07 02:28:29,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 02:28:29,427 [I] connection success
2026-08-07 02:28:29,427 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 02:28:29,427 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 02:28:29,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 02:28:29,680 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 02:28:29,680 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 02:28:29,680 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 02:28:29,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:38239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 02:28:29,949 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 02:28:30.466612 17620 mini_ranger_kms.cc:310] Created kms service
I20260807 02:28:30.701880 17620 mini_ranger_kms.cc:326] Created kudu user
I20260807 02:28:30.754159 17620 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 02:28:30 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:31.194466 17620 mini_ranger_kms.cc:384] Added ranger policy
I20260807 02:28:31.194662 17620 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 02:28:31.194697 17620 mini_ranger_kms.cc:240] Using host: 127.17.53.20
I20260807 02:28:31.200381 17620 mini_ranger_kms.cc:276] Ranger KMS PID: 1996
I20260807 02:28:31.200543 17620 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.53.20:47935
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 2:28:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 02:28:33 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786069713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:28:34.033252 17620 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.232s user 0.016s sys 0.255s
I20260807 02:28:34.033414 17620 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 02:28:34.033572 17620 mini_ranger_kms.cc:401] 127.17.53.20:47935/kms/v1/keys
Aug 07 02:28:34 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
Aug 07 02:28:34 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
Aug 07 02:28:37 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:28:37.236486 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34719
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:34719
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:47935/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:28:37.370368 2060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:28:37.370627 2060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:28:37.370667 2060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:28:37.374603 2060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:28:37.374662 2060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:28:37.374686 2060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:28:37.374703 2060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:28:37.374717 2060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:28:37.380147 2060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.53.20:47935/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:34719
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:28:37.381284 2060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:28:37.382273 2060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:28:37.391484 2072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:37.391469 2070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:28:37.391644 2060 server_base.cc:1034] running on GCE node
W20260807 02:28:37.391470 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:28:37.392354 2060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:28:37.393476 2060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:28:37.394663 2060 hybrid_clock.cc:648] HybridClock initialized: now 1786069717394641 us; error 32 us; skew 500 ppm
Aug 07 02:28:37 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069717, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:28:37.397460 2060 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:28:37.398447 2060 webserver.cc:460] Webserver started at http://127.17.53.62:44913/ using document root <none> and password file <none>
I20260807 02:28:37.399006 2060 fs_manager.cc:359] Metadata directory not provided
I20260807 02:28:37.399083 2060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:28:37.399382 2060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:28:37 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069717, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:37.449424 2060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "fb93d38b7cef4f23a14838c5278310f0"
format_stamp: "Formatted at 2026-08-07 02:28:37 on dist-test-slave-g170"
server_key: "cd1c8b11f853660efd4570ee493086d5"
server_key_iv: "228f7f3522243d8073593de0c4004294"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:37.450304 2060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "fb93d38b7cef4f23a14838c5278310f0"
format_stamp: "Formatted at 2026-08-07 02:28:37 on dist-test-slave-g170"
server_key: "cd1c8b11f853660efd4570ee493086d5"
server_key_iv: "228f7f3522243d8073593de0c4004294"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:37.455726 2060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 02:28:37.470093 2084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:37.471545 2060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:28:37.471694 2060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "fb93d38b7cef4f23a14838c5278310f0"
format_stamp: "Formatted at 2026-08-07 02:28:37 on dist-test-slave-g170"
server_key: "cd1c8b11f853660efd4570ee493086d5"
server_key_iv: "228f7f3522243d8073593de0c4004294"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:37.471845 2060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:28:37.489769 2060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:28:37.493141 2060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:28:37.493362 2060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:28:37.501720 2060 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:34719
I20260807 02:28:37.501739 2159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:34719 every 8 connection(s)
I20260807 02:28:37.502764 2060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:28:37.505745 2160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:28:37.506410 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 2060
I20260807 02:28:37.506546 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:28:37.506852 17620 external_mini_cluster.cc:1428] Setting key e736a13bd2794c24d76f5ac4631aacff
I20260807 02:28:37.512579 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: Bootstrap starting.
I20260807 02:28:37.515193 2160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: Neither blocks nor log segments found. Creating new log.
I20260807 02:28:37.516067 2160 log.cc:824] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:28:37 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:28:37.518494 2160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: No bootstrap required, opened a new log
I20260807 02:28:37.521747 2160 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } }
I20260807 02:28:37.521987 2160 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:28:37.522047 2160 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb93d38b7cef4f23a14838c5278310f0, State: Initialized, Role: FOLLOWER
I20260807 02:28:37.522557 2160 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } }
I20260807 02:28:37.522727 2160 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:28:37.522795 2160 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:28:37.522892 2160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:28:37.523484 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:37.508297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.17.53.62:34719)
0807 02:28:37.508777 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:37.508799 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:37.508855 (+ 56us) server_negotiation.cc:407] Connection header received
0807 02:28:37.509719 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:37.509743 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:37.510170 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:37.510537 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:28:37.511359 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:37.512848 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:37.513820 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:37.514492 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:37.514649 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:37.517526 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:28:37.517537 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:28:37.517554 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:28:37.517591 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:28:37.520278 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:37.520814 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:37.520825 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:37.520831 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:37.520955 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:37.521278 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:37.521283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:37.521285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:37.521716 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:28:37.521943 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:37.522325 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 02:28:37.522607 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260807 02:28:37.523878 2160 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } }
I20260807 02:28:37.524220 2160 leader_election.cc:302] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb93d38b7cef4f23a14838c5278310f0; no voters:
I20260807 02:28:37.524565 2160 leader_election.cc:288] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:28:37.524705 2166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:28:37.525061 2166 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [term 1 LEADER]: Becoming Leader. State: Replica: fb93d38b7cef4f23a14838c5278310f0, State: Running, Role: LEADER
I20260807 02:28:37.525450 2166 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } }
I20260807 02:28:37.525900 2160 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:28:37.526860 2167 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb93d38b7cef4f23a14838c5278310f0. Latest consensus state: current_term: 1 leader_uuid: "fb93d38b7cef4f23a14838c5278310f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } } }
I20260807 02:28:37.527004 2167 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [sys.catalog]: This master's current role is: LEADER
I20260807 02:28:37.527496 2181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:28:37.527451 2170 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb93d38b7cef4f23a14838c5278310f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb93d38b7cef4f23a14838c5278310f0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34719 } } }
I20260807 02:28:37.527556 2170 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0 [sys.catalog]: This master's current role is: LEADER
I20260807 02:28:37.530840 2181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:28:37.531314 2160 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 02:28:37.540889 2181 catalog_manager.cc:1347] Generated new cluster ID: 2d59413495a04bf49039a18466f2c5c6
I20260807 02:28:37.541013 2181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:28:37.560853 2181 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:28:37.562215 2181 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:28:37.570158 2181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: Generated new TSK 0
I20260807 02:28:37.570948 2181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:28:38.320996 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:47935/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:28:38.486128 2229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:28:38.486519 2229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:28:38.486634 2229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:28:38.492108 2229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:28:38.492203 2229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:28:38.492304 2229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:28:38.498019 2229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.53.20:47935/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:28:38.499130 2229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:28:38.500209 2229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:28:38.513226 2241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:38.513453 2245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:38.513293 2243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:28:38.514264 2229 server_base.cc:1034] running on GCE node
I20260807 02:28:38.514680 2229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:28:38.515389 2229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:28:38.516571 2229 hybrid_clock.cc:648] HybridClock initialized: now 1786069718516539 us; error 50 us; skew 500 ppm
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:28:38.520433 2229 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:28:38.521838 2229 webserver.cc:460] Webserver started at http://127.17.53.1:45001/ using document root <none> and password file <none>
I20260807 02:28:38.522706 2229 fs_manager.cc:359] Metadata directory not provided
I20260807 02:28:38.522825 2229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:28:38.523299 2229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:38.542516 2229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "7625a133376647a2b37ed1d20f31b496"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "44bf3754d4060b13225cef9c5f14ca06"
server_key_iv: "280317f8a251b2f9dc67305d1e84c88a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:38.543532 2229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "7625a133376647a2b37ed1d20f31b496"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "44bf3754d4060b13225cef9c5f14ca06"
server_key_iv: "280317f8a251b2f9dc67305d1e84c88a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:38.549264 2229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 02:28:38.589304 2255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:38.591343 2229 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.000s
I20260807 02:28:38.591507 2229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "7625a133376647a2b37ed1d20f31b496"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "44bf3754d4060b13225cef9c5f14ca06"
server_key_iv: "280317f8a251b2f9dc67305d1e84c88a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:38.591652 2229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:28:38.644191 2229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:28:38.648319 2229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:28:38.648634 2229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:28:38.649520 2229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:28:38.650640 2229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:28:38.650704 2229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:38.650758 2229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:28:38.650835 2229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 02:28:38.663251 2229 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:36629
I20260807 02:28:38.663461 2420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:36629 every 8 connection(s)
I20260807 02:28:38.664604 2229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:28:38.671864 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 2229
I20260807 02:28:38.672008 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:28:38.672376 17620 external_mini_cluster.cc:1428] Setting key 6e951d7efe2c21390876c5b6753ee02c
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:28:38.685592 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:38.668719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:42655 (local address 127.17.53.62:34719)
0807 02:28:38.669056 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:38.669063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:38.669082 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:28:38.669746 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:38.669750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:38.669822 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:38.669945 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:28:38.671696 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:38.672670 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:38.674664 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:38.675065 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:38.675194 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:38.678798 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:28:38.678802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:28:38.678805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:28:38.678833 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:28:38.681633 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:38.682333 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:38.682337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:38.682339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:38.682436 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:38.682871 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:38.682875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:38.682876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:38.683112 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:28:38.683208 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:38.685275 (+ 2067us) server_negotiation.cc:299] Negotiation successful
0807 02:28:38.685416 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":138,"threads_started":1}
I20260807 02:28:38.686537 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:38.667457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34719 (local address 127.17.53.1:42655)
0807 02:28:38.668128 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:38.668191 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:38.669437 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:38.670218 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:38.670229 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:38.670640 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:28:38.671450 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:38.671476 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:38.672870 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:38.672875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:38.674485 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:38.674498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:38.675397 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:38.675401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:38.675686 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:38.676493 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:28:38.676518 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:28:38.678576 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:28:38.681788 (+ 3212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:38.681794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:38.681813 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:38.682185 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:38.682551 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:38.682555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:38.682557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:38.682753 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:38.683279 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:28:38.683286 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:28:38.683629 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 02:28:38.683848 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 02:28:38.684205 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":193,"threads_started":1}
I20260807 02:28:38.688386 2421 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34719
I20260807 02:28:38.688643 2421 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:28:38.689311 2421 heartbeater.cc:511] Master 127.17.53.62:34719 requested a full tablet report, sending...
I20260807 02:28:38.691300 2109 ts_manager.cc:194] Registered new tserver with Master: 7625a133376647a2b37ed1d20f31b496 (127.17.53.1:36629)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:28:38.693089 2109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:42655
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:28:38.713388 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:47935/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:28:38.910604 2429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:28:38.911152 2429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:28:38.911301 2429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:28:38.918444 2429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:28:38.918691 2429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:28:38.918864 2429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:28:38.927961 2429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.53.20:47935/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:28:38.929769 2429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:28:38.931293 2429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:28:38.945598 2444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:38.945729 2441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:38.946508 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:28:38.947494 2429 server_base.cc:1034] running on GCE node
I20260807 02:28:38.948206 2429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:28:38.949172 2429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:28:38.950548 2429 hybrid_clock.cc:648] HybridClock initialized: now 1786069718950462 us; error 104 us; skew 500 ppm
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:28:38.958721 2429 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:28:38.960106 2429 webserver.cc:460] Webserver started at http://127.17.53.2:35247/ using document root <none> and password file <none>
I20260807 02:28:38.961057 2429 fs_manager.cc:359] Metadata directory not provided
I20260807 02:28:38.961225 2429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:28:38.961545 2429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:28:38 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:38.997212 2429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "c9ff0a8f886cb3a7b36ec85991e35b1b"
server_key_iv: "adaabf314b08bb49c5a2c1fddfa0413a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:38.998255 2429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "c9ff0a8f886cb3a7b36ec85991e35b1b"
server_key_iv: "adaabf314b08bb49c5a2c1fddfa0413a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:39.015692 2429 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.006s sys 0.000s
I20260807 02:28:39.041239 2457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.042886 2429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:28:39.043051 2429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12"
format_stamp: "Formatted at 2026-08-07 02:28:38 on dist-test-slave-g170"
server_key: "c9ff0a8f886cb3a7b36ec85991e35b1b"
server_key_iv: "adaabf314b08bb49c5a2c1fddfa0413a"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:39.043218 2429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:28:39.069361 2429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:28:39.073534 2429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:28:39.073762 2429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:28:39.074574 2429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:28:39.075776 2429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:28:39.075856 2429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.075933 2429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:28:39.075971 2429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.089130 2429 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:34731
I20260807 02:28:39.090451 2429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:28:39.097838 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 2429
I20260807 02:28:39.098024 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:28:39.098472 17620 external_mini_cluster.cc:1428] Setting key e3d520a5a246998d9944e273bbc97131
I20260807 02:28:39.107633 2622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:34731 every 8 connection(s)
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:28:39.133966 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.107345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60133 (local address 127.17.53.62:34719)
0807 02:28:39.107479 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.107484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.109513 (+ 2029us) server_negotiation.cc:407] Connection header received
0807 02:28:39.110949 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.110953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.111016 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.111076 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:28:39.114726 (+ 3650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.115685 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.117529 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.117890 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.117972 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.125489 (+ 7517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:28:39.125494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:28:39.125497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:28:39.125532 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:28:39.127906 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.128655 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.128660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.128662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.128904 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.129405 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.129409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.129411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.129742 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:28:39.129842 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.133550 (+ 3708us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.133752 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:28:39.135977 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.106183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34719 (local address 127.17.53.2:60133)
0807 02:28:39.109347 (+ 3164us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.109409 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.110642 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.111520 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.111530 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.111945 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:28:39.112678 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.112701 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.115895 (+ 3194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.115899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.117376 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.117389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.118206 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.118210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.118431 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.122640 (+ 4209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:28:39.122668 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:28:39.125235 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:28:39.128076 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:39.128082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:39.128103 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:39.128510 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:39.129045 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:39.129049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:39.129051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:39.129262 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:39.132694 (+ 3432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:28:39.132702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:28:39.133153 (+ 451us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.134289 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.134748 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2913,"thread_start_us":155,"threads_started":1}
I20260807 02:28:39.137634 2623 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34719
I20260807 02:28:39.137935 2623 heartbeater.cc:464] Registering TS with master...
I20260807 02:28:39.138680 2623 heartbeater.cc:511] Master 127.17.53.62:34719 requested a full tablet report, sending...
I20260807 02:28:39.143262 2109 ts_manager.cc:194] Registered new tserver with Master: d3e8ff9a82a140eca2ffa3d57cffdd12 (127.17.53.2:34731)
I20260807 02:28:39.144119 2109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:60133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:28:39.172390 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:47935/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
W20260807 02:28:39.445250 2632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:28:39.445669 2632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:28:39.445779 2632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:28:39.452559 2632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:28:39.452703 2632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:28:39.452843 2632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:28:39.461617 2632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.53.20:47935/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:28:39.463304 2632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:28:39.465194 2632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:28:39.479521 2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:39.479758 2642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:28:39.479554 2640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:28:39.483017 2632 server_base.cc:1034] running on GCE node
I20260807 02:28:39.483518 2632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:28:39.484383 2632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:28:39.485606 2632 hybrid_clock.cc:648] HybridClock initialized: now 1786069719485551 us; error 101 us; skew 500 ppm
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:28:39.489588 2632 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:28:39.490722 2632 webserver.cc:460] Webserver started at http://127.17.53.3:43431/ using document root <none> and password file <none>
I20260807 02:28:39.491523 2632 fs_manager.cc:359] Metadata directory not provided
I20260807 02:28:39.491669 2632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:28:39.491977 2632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:39.509523 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "658130f5f14d4861b4f5e2a595ac4897"
format_stamp: "Formatted at 2026-08-07 02:28:39 on dist-test-slave-g170"
server_key: "d2eb94c29432bbb6d5fae459dfe94449"
server_key_iv: "b45174b27eab7ddefa29db7148408bb9"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:39.510447 2632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "658130f5f14d4861b4f5e2a595ac4897"
format_stamp: "Formatted at 2026-08-07 02:28:39 on dist-test-slave-g170"
server_key: "d2eb94c29432bbb6d5fae459dfe94449"
server_key_iv: "b45174b27eab7ddefa29db7148408bb9"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:39.516300 2632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 02:28:39.547289 2651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.549755 2632 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260807 02:28:39.549929 2632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "658130f5f14d4861b4f5e2a595ac4897"
format_stamp: "Formatted at 2026-08-07 02:28:39 on dist-test-slave-g170"
server_key: "d2eb94c29432bbb6d5fae459dfe94449"
server_key_iv: "b45174b27eab7ddefa29db7148408bb9"
server_key_version: "kuduclusterkey@0"
I20260807 02:28:39.550096 2632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:28:39.573309 2632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:28:39.577370 2632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:28:39.577657 2632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:28:39.578507 2632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:28:39.579830 2632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:28:39.579942 2632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.580062 2632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:28:39.580137 2632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:28:39.599370 2632 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:33017
I20260807 02:28:39.600708 2632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:28:39.602785 2805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:33017 every 8 connection(s)
I20260807 02:28:39.606468 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 2632
I20260807 02:28:39.606592 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:28:39.606911 17620 external_mini_cluster.cc:1428] Setting key f8c1bee8be18919cffd0ce73f5c36e63
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069719, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:28:39.622851 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.605057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:50663 (local address 127.17.53.62:34719)
0807 02:28:39.605191 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.605196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.607676 (+ 2480us) server_negotiation.cc:407] Connection header received
0807 02:28:39.609090 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.609094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.609159 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.609235 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:28:39.611045 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.611995 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.613946 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.614346 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.614433 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.617856 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:28:39.617861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:28:39.617864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:28:39.617899 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:28:39.619952 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.620686 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.620690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.620692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.620845 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.621363 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.621367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.621370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.621615 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:28:39.621716 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.622532 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.622673 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:28:39.624644 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.605674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34719 (local address 127.17.53.3:50663)
0807 02:28:39.607490 (+ 1816us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.607556 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.608809 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.609693 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.609703 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.610101 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:28:39.610853 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.610878 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.612169 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.612172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.613766 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.613779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.614659 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.614663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.614884 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.615687 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:28:39.615710 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:28:39.617634 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:28:39.620120 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:39.620126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:39.620143 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:39.620541 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:39.621000 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:28:39.621004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:28:39.621007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:28:39.621225 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:28:39.621799 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:28:39.621806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:28:39.622159 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.623110 (+ 951us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.623490 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":163,"threads_started":1}
I20260807 02:28:39.626004 2806 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34719
I20260807 02:28:39.626307 2806 heartbeater.cc:464] Registering TS with master...
I20260807 02:28:39.627007 2806 heartbeater.cc:511] Master 127.17.53.62:34719 requested a full tablet report, sending...
I20260807 02:28:39.628227 2109 ts_manager.cc:194] Registered new tserver with Master: 658130f5f14d4861b4f5e2a595ac4897 (127.17.53.3:33017)
I20260807 02:28:39.629071 2109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:50663
I20260807 02:28:39.632242 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:28:39.644237 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.634610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.17.53.62:34719)
0807 02:28:39.634716 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.634720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.635327 (+ 607us) server_negotiation.cc:407] Connection header received
0807 02:28:39.635496 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.635500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.635542 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.635589 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:28:39.636508 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.637299 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.638481 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.638803 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.638874 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.640639 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:28:39.640643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:28:39.640646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:28:39.640671 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:28:39.642377 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.642875 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.642878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.642880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.642968 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:28:39.643270 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:28:39.643273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:28:39.643275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:28:39.643479 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:28:39.643576 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.643957 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.644079 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 02:28:39.648253 2109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:28:39.653337 2109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:28:39.687309 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.667891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33017 (local address 127.0.0.1:51350)
0807 02:28:39.672923 (+ 5032us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.672936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.673028 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.674308 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.674314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.674340 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:28:39.674473 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.674480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.678063 (+ 3583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.678066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.679442 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.679454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.686691 (+ 7237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.686694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.686812 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.686853 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.686972 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.687123 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4968,"thread_start_us":89,"threads_started":1}
I20260807 02:28:39.687309 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.670084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36629 (local address 127.0.0.1:32874)
0807 02:28:39.674074 (+ 3990us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.674088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.674177 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.675809 (+ 1632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.675812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.675829 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:28:39.675927 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.675932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.680271 (+ 4339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.680274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.681458 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.681470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.686918 (+ 5448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.686924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.687026 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.687047 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.687116 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.687169 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3879,"thread_start_us":91,"threads_started":1}
I20260807 02:28:39.690246 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.665400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:34731 (local address 127.0.0.1:54878)
0807 02:28:39.671132 (+ 5732us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.671184 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.671327 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.674835 (+ 3508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.674839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.674855 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:28:39.674952 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.674958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.682375 (+ 7417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.682378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.684303 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.684314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.688747 (+ 4433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.688751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.689928 (+ 1177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.689955 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.690031 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.690108 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5544,"thread_start_us":116,"threads_started":1}
I20260807 02:28:39.690999 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.670462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32874 (local address 127.17.53.1:36629)
0807 02:28:39.671812 (+ 1350us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.671818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.674303 (+ 2485us) server_negotiation.cc:407] Connection header received
0807 02:28:39.674375 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.674380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.674569 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.674692 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:28:39.676051 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.680141 (+ 4090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.684437 (+ 4296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.685289 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.685395 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.685550 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.690704 (+ 5154us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.690848 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1251,"thread_start_us":114,"threads_started":1}
I20260807 02:28:39.691540 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.668315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.17.53.3:33017)
0807 02:28:39.671623 (+ 3308us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.671634 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.673402 (+ 1768us) server_negotiation.cc:407] Connection header received
0807 02:28:39.673483 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.673488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.673658 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.673813 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:28:39.676333 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.677827 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.685730 (+ 7903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.686572 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.686690 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.686854 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.691258 (+ 4404us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.691377 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3199,"thread_start_us":99,"threads_started":1}
I20260807 02:28:39.692051 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.666397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.17.53.2:34731)
0807 02:28:39.672720 (+ 6323us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.672727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.672752 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:28:39.672875 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.672881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.673119 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.673256 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:28:39.678374 (+ 5118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.682232 (+ 3858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.687368 (+ 5136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.688451 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.688556 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.688687 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.691803 (+ 3116us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.691918 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6208,"thread_start_us":72,"threads_started":1}
I20260807 02:28:39.696415 2421 heartbeater.cc:503] Master 127.17.53.62:34719 was elected leader, sending a full tablet report...
I20260807 02:28:39.695812 2312 tablet_service.cc:1467] Processing CreateTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 (DEFAULT_TABLE table=test-table [id=258207d922e24ed2bff27ab359109f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:28:39.699290 2312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c246487efd440dea5f1a47f8b67dd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:28:39.708070 2712 tablet_service.cc:1467] Processing CreateTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 (DEFAULT_TABLE table=test-table [id=258207d922e24ed2bff27ab359109f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:28:39.709336 2712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c246487efd440dea5f1a47f8b67dd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:28:39.710600 2829 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Bootstrap starting.
I20260807 02:28:39.713198 2829 tablet_bootstrap.cc:654] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Neither blocks nor log segments found. Creating new log.
I20260807 02:28:39.714214 2829 log.cc:824] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Log is configured to *not* fsync() on all Append() calls
I20260807 02:28:39.716709 2829 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: No bootstrap required, opened a new log
I20260807 02:28:39.716957 2829 ts_tablet_manager.cc:1397] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 02:28:39.721891 2516 tablet_service.cc:1467] Processing CreateTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 (DEFAULT_TABLE table=test-table [id=258207d922e24ed2bff27ab359109f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:28:39.722262 2829 raft_consensus.cc:348] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.722584 2829 raft_consensus.cc:374] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:28:39.722652 2829 raft_consensus.cc:729] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7625a133376647a2b37ed1d20f31b496, State: Initialized, Role: FOLLOWER
I20260807 02:28:39.723191 2516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c246487efd440dea5f1a47f8b67dd17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:28:39.723335 2829 consensus_queue.cc:260] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.724663 2829 ts_tablet_manager.cc:1428] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 02:28:39.748605 2835 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Bootstrap starting.
I20260807 02:28:39.749547 2838 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Bootstrap starting.
I20260807 02:28:39.752417 2838 tablet_bootstrap.cc:654] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Neither blocks nor log segments found. Creating new log.
I20260807 02:28:39.753407 2838 log.cc:824] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Log is configured to *not* fsync() on all Append() calls
I20260807 02:28:39.755749 2838 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: No bootstrap required, opened a new log
I20260807 02:28:39.755971 2838 ts_tablet_manager.cc:1397] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 02:28:39.757131 2835 tablet_bootstrap.cc:654] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Neither blocks nor log segments found. Creating new log.
I20260807 02:28:39.758909 2835 log.cc:824] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:28:39 dist-test-slave-g170 krb5kdc[1419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069718, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:28:39.761620 2838 raft_consensus.cc:348] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.761936 2838 raft_consensus.cc:374] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:28:39.761987 2838 raft_consensus.cc:729] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e8ff9a82a140eca2ffa3d57cffdd12, State: Initialized, Role: FOLLOWER
I20260807 02:28:39.762487 2838 consensus_queue.cc:260] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.763607 2838 ts_tablet_manager.cc:1428] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 02:28:39.764003 2623 heartbeater.cc:503] Master 127.17.53.62:34719 was elected leader, sending a full tablet report...
I20260807 02:28:39.771307 2835 tablet_bootstrap.cc:492] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: No bootstrap required, opened a new log
I20260807 02:28:39.771656 2835 ts_tablet_manager.cc:1397] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.006s
I20260807 02:28:39.776746 2835 raft_consensus.cc:348] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.777192 2835 raft_consensus.cc:374] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:28:39.777307 2835 raft_consensus.cc:729] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658130f5f14d4861b4f5e2a595ac4897, State: Initialized, Role: FOLLOWER
I20260807 02:28:39.777933 2835 consensus_queue.cc:260] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.779165 2806 heartbeater.cc:503] Master 127.17.53.62:34719 was elected leader, sending a full tablet report...
I20260807 02:28:39.779346 2835 ts_tablet_manager.cc:1428] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260807 02:28:39.846555 2844 raft_consensus.cc:482] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:28:39.846828 2844 raft_consensus.cc:504] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.848344 2844 leader_election.cc:288] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 658130f5f14d4861b4f5e2a595ac4897 (127.17.53.3:33017), 7625a133376647a2b37ed1d20f31b496 (127.17.53.1:36629)
W20260807 02:28:39.854398 2624 tablet.cc:2369] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:28:39.854996 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.849742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:59315 (local address 127.17.53.1:36629)
0807 02:28:39.849864 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.849868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.850221 (+ 353us) server_negotiation.cc:407] Connection header received
0807 02:28:39.850453 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.850457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.850512 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.850562 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:28:39.851010 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.851804 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.853605 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.854241 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.854325 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.854398 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.854773 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.854861 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:28:39.855474 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.848808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36629 (local address 127.17.53.2:59315)
0807 02:28:39.850113 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.850129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.850316 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.850708 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.850711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.850749 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:28:39.850886 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.850892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.851952 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.851955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.853437 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.853451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.854502 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.854505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.854621 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.854647 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.855237 (+ 590us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.855326 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"thread_start_us":141,"threads_started":1}
I20260807 02:28:39.856178 2326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c246487efd440dea5f1a47f8b67dd17" candidate_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7625a133376647a2b37ed1d20f31b496" is_pre_election: true
I20260807 02:28:39.856539 2326 raft_consensus.cc:2454] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e8ff9a82a140eca2ffa3d57cffdd12 in term 0.
I20260807 02:28:39.857121 2458 leader_election.cc:302] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7625a133376647a2b37ed1d20f31b496, d3e8ff9a82a140eca2ffa3d57cffdd12; no voters:
W20260807 02:28:39.857303 2807 tablet.cc:2369] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:28:39.857470 2844 raft_consensus.cc:2781] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:28:39.857544 2844 raft_consensus.cc:482] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:28:39.857586 2844 raft_consensus.cc:3037] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:28:39.859068 2844 raft_consensus.cc:504] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.859494 2844 leader_election.cc:288] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [CANDIDATE]: Term 1 election: Requested vote from peers 658130f5f14d4861b4f5e2a595ac4897 (127.17.53.3:33017), 7625a133376647a2b37ed1d20f31b496 (127.17.53.1:36629)
I20260807 02:28:39.859990 2326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c246487efd440dea5f1a47f8b67dd17" candidate_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7625a133376647a2b37ed1d20f31b496"
I20260807 02:28:39.860124 2326 raft_consensus.cc:3037] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:28:39.861680 2326 raft_consensus.cc:2454] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e8ff9a82a140eca2ffa3d57cffdd12 in term 1.
I20260807 02:28:39.861821 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.849471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:53861 (local address 127.17.53.3:33017)
0807 02:28:39.849612 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.849617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.857261 (+ 7644us) server_negotiation.cc:407] Connection header received
0807 02:28:39.857484 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.857487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.857547 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.857602 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:28:39.858061 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.858848 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.860400 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.861050 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.861136 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.861203 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.861584 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.861686 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:28:39.862047 2458 leader_election.cc:302] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7625a133376647a2b37ed1d20f31b496, d3e8ff9a82a140eca2ffa3d57cffdd12; no voters:
I20260807 02:28:39.862247 2844 raft_consensus.cc:2781] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:28:39.862504 2844 raft_consensus.cc:686] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 1 LEADER]: Becoming Leader. State: Replica: d3e8ff9a82a140eca2ffa3d57cffdd12, State: Running, Role: LEADER
I20260807 02:28:39.862900 2844 consensus_queue.cc:237] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } }
I20260807 02:28:39.862254 2849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.849304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:33017 (local address 127.17.53.2:53861)
0807 02:28:39.857152 (+ 7848us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:39.857169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:28:39.857344 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:28:39.857774 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:28:39.857776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:28:39.857796 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:28:39.857923 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.857930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.859030 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.859034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.860225 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:28:39.860237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.861316 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.861319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:28:39.861437 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.861460 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:28:39.862030 (+ 570us) client_negotiation.cc:240] Negotiation successful
0807 02:28:39.862110 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7770,"thread_start_us":109,"threads_started":1}
I20260807 02:28:39.864070 2743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c246487efd440dea5f1a47f8b67dd17" candidate_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658130f5f14d4861b4f5e2a595ac4897" is_pre_election: true
I20260807 02:28:39.864389 2743 raft_consensus.cc:2454] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e8ff9a82a140eca2ffa3d57cffdd12 in term 0.
I20260807 02:28:39.864742 2741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c246487efd440dea5f1a47f8b67dd17" candidate_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658130f5f14d4861b4f5e2a595ac4897"
I20260807 02:28:39.864884 2741 raft_consensus.cc:3037] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:28:39.866618 2741 raft_consensus.cc:2454] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e8ff9a82a140eca2ffa3d57cffdd12 in term 1.
I20260807 02:28:39.870098 2109 catalog_manager.cc:5495] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3e8ff9a82a140eca2ffa3d57cffdd12 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e8ff9a82a140eca2ffa3d57cffdd12" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 34731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 } health_report { overall_health: UNKNOWN } } }
W20260807 02:28:39.917711 2422 tablet.cc:2369] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:28:39.931137 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:39.925585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.17.53.2:34731)
0807 02:28:39.925742 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:28:39.925747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:28:39.925813 (+ 66us) server_negotiation.cc:407] Connection header received
0807 02:28:39.926004 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:28:39.926008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:28:39.926087 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:28:39.926207 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:28:39.926621 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.927426 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.928719 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:28:39.929114 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:28:39.929226 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:28:39.929698 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:28:39.930298 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:28:39.930427 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 02:28:39.930925 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 02:28:39.930989 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:28:39.941819 2326 raft_consensus.cc:1261] T 0c246487efd440dea5f1a47f8b67dd17 P 7625a133376647a2b37ed1d20f31b496 [term 1 FOLLOWER]: Refusing update from remote peer d3e8ff9a82a140eca2ffa3d57cffdd12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:28:39.942783 2850 consensus_queue.cc:1035] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7625a133376647a2b37ed1d20f31b496" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 36629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:28:39.946084 2741 raft_consensus.cc:1261] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 1 FOLLOWER]: Refusing update from remote peer d3e8ff9a82a140eca2ffa3d57cffdd12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:28:39.946933 2850 consensus_queue.cc:1035] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "658130f5f14d4861b4f5e2a595ac4897" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:28:39.966188 2854 mvcc.cc:205] Tried to move back new op lower bound from 7315741572872732672 to 7315741572576415744. Current Snapshot: MvccSnapshot[applied={T|T < 7315741572872732672}]
I20260807 02:28:39.970881 2855 mvcc.cc:205] Tried to move back new op lower bound from 7315741572872732672 to 7315741572576415744. Current Snapshot: MvccSnapshot[applied={T|T < 7315741572872732672}]
I20260807 02:28:39.983678 2109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:28:39.983878 2109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:28:39.986951 2109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb93d38b7cef4f23a14838c5278310f0: Sending DeleteTablet for 3 replicas of tablet 0c246487efd440dea5f1a47f8b67dd17
I20260807 02:28:39.988137 2312 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:28:39 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:32874
I20260807 02:28:39.989133 2712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:28:39 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:51350
I20260807 02:28:39.989732 2516 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c246487efd440dea5f1a47f8b67dd17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:28:39 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:54878
I20260807 02:28:39.990923 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 2229
I20260807 02:28:40.005033 2865 tablet_replica.cc:330] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: stopping tablet replica
I20260807 02:28:40.005513 2865 raft_consensus.cc:2229] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:28:40.005851 2865 raft_consensus.cc:2258] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:28:40.006229 2866 tablet_replica.cc:330] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: stopping tablet replica
I20260807 02:28:40.006641 2866 raft_consensus.cc:2229] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:28:40.007170 2866 raft_consensus.cc:2258] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:28:40.010673 2866 ts_tablet_manager.cc:1905] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:28:40.014142 2866 ts_tablet_manager.cc:1918] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:28:40.014266 2866 log.cc:1196] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/0c246487efd440dea5f1a47f8b67dd17
I20260807 02:28:40.014605 2866 ts_tablet_manager.cc:1939] T 0c246487efd440dea5f1a47f8b67dd17 P d3e8ff9a82a140eca2ffa3d57cffdd12: Deleting consensus metadata
I20260807 02:28:40.016093 2865 ts_tablet_manager.cc:1905] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:28:40.016084 2085 catalog_manager.cc:4841] TS d3e8ff9a82a140eca2ffa3d57cffdd12 (127.17.53.2:34731): tablet 0c246487efd440dea5f1a47f8b67dd17 (table test-table [id=258207d922e24ed2bff27ab359109f62]) successfully deleted
I20260807 02:28:40.019093 2865 ts_tablet_manager.cc:1918] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:28:40.019227 2865 log.cc:1196] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/0c246487efd440dea5f1a47f8b67dd17
I20260807 02:28:40.019610 2865 ts_tablet_manager.cc:1939] T 0c246487efd440dea5f1a47f8b67dd17 P 658130f5f14d4861b4f5e2a595ac4897: Deleting consensus metadata
I20260807 02:28:40.021086 2086 catalog_manager.cc:4841] TS 658130f5f14d4861b4f5e2a595ac4897 (127.17.53.3:33017): tablet 0c246487efd440dea5f1a47f8b67dd17 (table test-table [id=258207d922e24ed2bff27ab359109f62]) successfully deleted
W20260807 02:28:40.024732 2085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:36629 (error 108)
I20260807 02:28:40.035666 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:28:40.035348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:36629 (local address 127.0.0.1:32880)
0807 02:28:40.035481 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 02:28:40.035566 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:36629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 02:28:40.035897 2085 catalog_manager.cc:4584] TS 7625a133376647a2b37ed1d20f31b496 (127.17.53.1:36629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c246487efd440dea5f1a47f8b67dd17: Network error: Client connection negotiation failed: client connection to 127.17.53.1:36629: connect: Connection refused (error 111)
I20260807 02:28:40.036265 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 2429
I20260807 02:28:40.072490 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 2632
I20260807 02:28:40.081959 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 2060
I20260807 02:28:40.104331 17620 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 02:28:40.450197 17620 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 02:28:40.450325 17620 mini_ranger.cc:66] Stopping Ranger...
I20260807 02:28:40.804932 17620 mini_ranger.cc:68] Stopped Ranger
2026-08-07 02:28:40.805 UTC [1456] LOG: received smart shutdown request
2026-08-07 02:28:40.809 UTC [1456] LOG: background worker "logical replication launcher" (PID 1474) exited with exit code 1
2026-08-07 02:28:40.809 UTC [1465] LOG: shutting down
2026-08-07 02:28:40.992 UTC [1456] LOG: database system is shut down
2026-08-07T02:28:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50680 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:28:41 dist-test-slave-g170 krb5kdc[2889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:28:41 dist-test-slave-g170 krb5kdc[2889](info): set up 2 sockets
Aug 07 02:28:41 dist-test-slave-g170 krb5kdc[2889](info): commencing operation
krb5kdc: starting...
W20260807 02:28:42.401487 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:28:42 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:28:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:28:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangeradmin_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangeradmin_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangerlookup_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangerlookup_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
I20260807 02:28:42.587054 17620 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPRMQP4/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/postgres -l logfile start
2026-08-07 02:28:45.203 UTC [2933] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 02:28:45.204 UTC [2933] LOG: listening on IPv4 address "127.17.53.20", port 46823
2026-08-07 02:28:45.208 UTC [2933] LOG: listening on Unix socket "/tmp/.s.PGSQL.46823"
2026-08-07 02:28:45.247 UTC [2936] LOG: database system was shut down at 2026-08-07 02:28:43 UTC
2026-08-07 02:28:45.252 UTC [2933] LOG: database system is ready to accept connections
I20260807 02:28:46.460086 17620 mini_postgres.cc:96] Postgres bound to 46823
2026-08-07 02:28:46.467 UTC [2945] FATAL: database "slave" does not exist
127.17.53.20:46823 - accepting connections
I20260807 02:28:46.468621 17620 mini_ranger.cc:161] Starting Ranger...
I20260807 02:28:46.485419 17620 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 02:28:47.013809 17620 mini_ranger.cc:87] Created ranger Postgres database
I20260807 02:28:47.013911 17620 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin
2026-08-07 02:28:47,288 [I] DB FLAVOR :POSTGRES
2026-08-07 02:28:47,289 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 02:28:47,289 [I] Checking connection..
2026-08-07 02:28:47,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 02:28:47,543 [I] Checking connection passed.
2026-08-07 02:28:47,543 [I] --------- Verifying version history table ---------
2026-08-07 02:28:47,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:28:47,831 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 02:28:47,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:28:48,112 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 02:28:48,112 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 02:28:48,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 02:28:48.370 UTC [3037] WARNING: there is no transaction in progress
2026-08-07 02:28:48,380 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 02:28:48,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 02:28:48,645 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 02:28:48,645 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 02:28:48,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 02:28:48,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 02:28:49,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-g170.c.gcp-upstream.internal','N') ;"
2026-08-07 02:28:49,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 02:28:49,685 [I] Table x_portal_user does not exist in database ranger
2026-08-07 02:28:49,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 02:28:49,968 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 02:28:49,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 02:28:50,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 02:28:50,511 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 02:28:50,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 02:28:51.530 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.547 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.564 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.580 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.597 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.615 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.679 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.686 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.700 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.711 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.721 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.737 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.748 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.755 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.764 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:51.771 UTC [3184] WARNING: there is no transaction in progress
2026-08-07 02:28:52,448 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 02:28:52,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 02:28:52,708 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 02:28:52,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 02:28:52,977 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 02:28:52,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 02:28:53,244 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 02:28:53,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 02:28:53,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 02:28:53,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-g170.c.gcp-upstream.internal';"
2026-08-07 02:28:53,989 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 02:28:53,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 02:28:54,267 [I] DB_PATCHES have already been applied
I20260807 02:28:54.277226 17620 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPRMQP4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 02:28:54.277318 17620 mini_ranger.cc:193] Using host: 127.17.53.20
I20260807 02:28:54.282313 17620 mini_ranger.cc:227] Ranger admin URL: http://127.17.53.20:46297
Aug 07, 2026 2:28:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 2:28:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 2:28:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 2:28:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 2:28:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 2:28:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 2:28:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 02:28:55.689970 17620 mini_ranger.cc:160] Time spent starting Ranger: real 9.221s user 0.006s sys 0.008s
Aug 07, 2026 2:29:16 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 2:29:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 2:29:16 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 2:29:16 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 2:29:17 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 02:29:18.132372 17620 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.53.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.53.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangerkms_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.53.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/rangerkms_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.53.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.53.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.53.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/HTTP_127.17.53.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 02:29:18 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069758, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 02:29:18.223742 17620 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 02:29:18.235700 17620 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 02:29:19.199246 17620 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 02:29:19.199342 17620 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms
I20260807 02:29:19.199370 17620 mini_ranger_kms.cc:227] Using postgres at 127.17.53.20:46823
2026-08-07 02:29:19,379 [I] DB FLAVOR :POSTGRES
2026-08-07 02:29:19,379 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 02:29:19,379 [I] Checking connection
2026-08-07 02:29:19,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 02:29:19,629 [I] connection success
2026-08-07 02:29:19,629 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 02:29:19,629 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 02:29:19,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 02:29:19,886 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 02:29:19,886 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 02:29:19,886 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 02:29:19,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.53.20:46823/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 02:29:20,175 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 02:29:20.689246 17620 mini_ranger_kms.cc:310] Created kms service
I20260807 02:29:20.896819 17620 mini_ranger_kms.cc:326] Created kudu user
I20260807 02:29:20.948738 17620 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 02:29:20 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069758, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:21.382431 17620 mini_ranger_kms.cc:384] Added ranger policy
I20260807 02:29:21.382696 17620 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPRMQP4/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPRMQP4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 02:29:21.382776 17620 mini_ranger_kms.cc:240] Using host: 127.17.53.20
I20260807 02:29:21.387499 17620 mini_ranger_kms.cc:276] Ranger KMS PID: 3447
I20260807 02:29:21.387589 17620 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.53.20:54987
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 2:29:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 02:29:23 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786069763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:24.188028 17620 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.964s user 0.004s sys 0.162s
I20260807 02:29:24.188185 17620 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 02:29:24.188267 17620 mini_ranger_kms.cc:401] 127.17.53.20:54987/kms/v1/keys
Aug 07 02:29:24 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
Aug 07 02:29:24 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.53.20@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
Aug 07 02:29:27 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:29:27.056424 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36431
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36431
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:27.175657 3521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:27.175945 3521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:27.175987 3521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:27.180081 3521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:27.180145 3521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:27.180166 3521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:27.180181 3521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:27.180197 3521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:27.185667 3521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.53.20:54987/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36431
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:27.186687 3521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:27.187685 3521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:29:27.197266 3521 server_base.cc:1034] running on GCE node
W20260807 02:29:27.197225 3529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:27.197172 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:27.197196 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:27.198343 3521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:27.199674 3521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:27.200860 3521 hybrid_clock.cc:648] HybridClock initialized: now 1786069767200822 us; error 51 us; skew 500 ppm
Aug 07 02:29:27 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:27.207795 3521 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:27.208966 3521 webserver.cc:460] Webserver started at http://127.17.53.62:37221/ using document root <none> and password file <none>
I20260807 02:29:27.209575 3521 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:27.209638 3521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:27.209830 3521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:29:27 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:27.263928 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "597b0ac1d41940bdb41ac9a9d0edd736"
format_stamp: "Formatted at 2026-08-07 02:29:27 on dist-test-slave-g170"
server_key: "6b7bb147f064099b582431a7553b6f8d"
server_key_iv: "fff8bc1b13da5bf748119446474b59ce"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:27.264659 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "597b0ac1d41940bdb41ac9a9d0edd736"
format_stamp: "Formatted at 2026-08-07 02:29:27 on dist-test-slave-g170"
server_key: "6b7bb147f064099b582431a7553b6f8d"
server_key_iv: "fff8bc1b13da5bf748119446474b59ce"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:27.269414 3521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:29:27.285128 3544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:27.286720 3521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:29:27.286911 3521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "597b0ac1d41940bdb41ac9a9d0edd736"
format_stamp: "Formatted at 2026-08-07 02:29:27 on dist-test-slave-g170"
server_key: "6b7bb147f064099b582431a7553b6f8d"
server_key_iv: "fff8bc1b13da5bf748119446474b59ce"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:27.287071 3521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:27.310806 3521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:27.314072 3521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:27.314312 3521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:27.323575 3521 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36431
I20260807 02:29:27.323581 3626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36431 every 8 connection(s)
I20260807 02:29:27.324703 3521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:27.325558 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 3521
I20260807 02:29:27.325698 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:29:27.326005 17620 external_mini_cluster.cc:1428] Setting key 41519b6dda4e23b1720e1b8d7f1145a7
I20260807 02:29:27.329320 3627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 02:29:27 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:27.336721 3627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Bootstrap starting.
I20260807 02:29:27.339699 3627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:27.341318 3627 log.cc:824] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:27.341449 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:27.327612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.17.53.62:36431)
0807 02:29:27.328114 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:27.328134 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:27.328184 (+ 50us) server_negotiation.cc:407] Connection header received
0807 02:29:27.328967 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:27.328987 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:27.329368 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:27.329796 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:27.330694 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:27.331791 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:27.332759 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:27.333311 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:27.333511 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:27.336295 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:27.336301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:27.336315 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:27.336345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:27.338195 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:27.338716 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:27.338722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:27.338727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:27.338847 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:27.339170 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:27.339174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:27.339176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:27.339596 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:27.339843 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:27.340159 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 02:29:27.340492 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":161,"threads_started":1}
I20260807 02:29:27.343763 3627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: No bootstrap required, opened a new log
I20260807 02:29:27.347360 3627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:27.347591 3627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:27.347669 3627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597b0ac1d41940bdb41ac9a9d0edd736, State: Initialized, Role: FOLLOWER
I20260807 02:29:27.348117 3627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:27.348245 3627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:27.348346 3627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:27.348453 3627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:27.349509 3627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:27.349888 3627 leader_election.cc:302] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597b0ac1d41940bdb41ac9a9d0edd736; no voters:
I20260807 02:29:27.350276 3627 leader_election.cc:288] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:29:27.350363 3634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:27.350631 3634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 LEADER]: Becoming Leader. State: Replica: 597b0ac1d41940bdb41ac9a9d0edd736, State: Running, Role: LEADER
I20260807 02:29:27.351446 3634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:27.351725 3627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:27.353135 3636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597b0ac1d41940bdb41ac9a9d0edd736. Latest consensus state: current_term: 1 leader_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } } }
I20260807 02:29:27.353272 3636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:27.353601 3635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } } }
I20260807 02:29:27.353679 3635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:27.354053 3644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:27.354216 3627 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 02:29:27.360338 3644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:27.369516 3644 catalog_manager.cc:1347] Generated new cluster ID: 34f4c009743e4e63a12f3a2c378a5759
I20260807 02:29:27.369582 3644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:27.384737 3644 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:29:27.386108 3644 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:27.397666 3644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Generated new TSK 0
I20260807 02:29:27.398486 3644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:29:28.116565 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:28.279551 3685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:28.279811 3685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:28.279855 3685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:28.284004 3685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:28.284067 3685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:28.284127 3685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:28.290664 3685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:28.291968 3685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:28.293146 3685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:28.303714 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:28.303890 3697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:28.304198 3695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:28.305441 3685 server_base.cc:1034] running on GCE node
I20260807 02:29:28.305878 3685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:28.306627 3685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:28.307821 3685 hybrid_clock.cc:648] HybridClock initialized: now 1786069768307784 us; error 49 us; skew 500 ppm
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:28.311112 3685 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:28.312368 3685 webserver.cc:460] Webserver started at http://127.17.53.1:46561/ using document root <none> and password file <none>
I20260807 02:29:28.313112 3685 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:28.313216 3685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:28.313489 3685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:28.339265 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "18a5a3f9a7b0400ca91c51c6b1667586"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "3597e11fc67ffad2b4b3c3591ba88b1e"
server_key_iv: "b495d6b6ccecc0a73ba5908398f82995"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.340173 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "18a5a3f9a7b0400ca91c51c6b1667586"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "3597e11fc67ffad2b4b3c3591ba88b1e"
server_key_iv: "b495d6b6ccecc0a73ba5908398f82995"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.350236 3685 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260807 02:29:28.367828 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.369661 3685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:29:28.369819 3685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "18a5a3f9a7b0400ca91c51c6b1667586"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "3597e11fc67ffad2b4b3c3591ba88b1e"
server_key_iv: "b495d6b6ccecc0a73ba5908398f82995"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.369966 3685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:28.387421 3685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:28.391348 3685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:28.391613 3685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:28.392401 3685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:28.393568 3685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:28.393675 3685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.393819 3685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:28.393908 3685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.407025 3685 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45165
I20260807 02:29:28.409647 3685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:28.412439 3872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45165 every 8 connection(s)
I20260807 02:29:28.419909 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 3685
I20260807 02:29:28.420061 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:29:28.420405 17620 external_mini_cluster.cc:1428] Setting key 1fbdcb35ec55d0f89e99e9733182a134
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:28.433733 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:28.414549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:55827 (local address 127.17.53.62:36431)
0807 02:29:28.414831 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:28.414837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:28.415935 (+ 1098us) server_negotiation.cc:407] Connection header received
0807 02:29:28.417444 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:28.417448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:28.417511 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:28.417583 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:28.419520 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.420478 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.423380 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.423767 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.423861 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:28.428726 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:28.428733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:28.428739 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:28.428819 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:28.431048 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:28.431697 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:28.431702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:28.431704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:28.431815 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:28.432247 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:28.432252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:28.432254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:28.432491 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:28.432576 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:28.433387 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 02:29:28.433520 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260807 02:29:28.435444 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:28.415047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.1:55827)
0807 02:29:28.415749 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:28.415813 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:28.417142 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:28.418035 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:28.418044 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:28.418519 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:28.419292 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:28.419315 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.420665 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.420669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:28.423221 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:28.423234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.424098 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.424102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:28.424349 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:28.426513 (+ 2164us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:28.426539 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:28.428477 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:28.431204 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:28.431212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:28.431233 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:28.431572 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:28.431942 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:28.431944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:28.431946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:28.432125 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:28.432649 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:28.432657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:28.433073 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 02:29:28.433980 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 02:29:28.434336 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":161,"threads_started":1}
I20260807 02:29:28.436923 3875 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:28.437201 3875 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:28.437806 3875 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:28.439786 3569 ts_manager.cc:194] Registered new tserver with Master: 18a5a3f9a7b0400ca91c51c6b1667586 (127.17.53.1:45165)
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:29:28.441570 3569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:55827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:29:28.468991 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 02:29:28.639389 3882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:28.639894 3882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:28.640005 3882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:28.646234 3882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:28.646369 3882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:28.646487 3882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:28.654490 3882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:28.655979 3882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:28.657445 3882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:28.670578 3893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:28.670830 3888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:28.671332 3890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:28.671921 3882 server_base.cc:1034] running on GCE node
I20260807 02:29:28.672480 3882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:28.673434 3882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:28.674654 3882 hybrid_clock.cc:648] HybridClock initialized: now 1786069768674630 us; error 42 us; skew 500 ppm
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:28.678915 3882 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:28.680173 3882 webserver.cc:460] Webserver started at http://127.17.53.2:41167/ using document root <none> and password file <none>
I20260807 02:29:28.680968 3882 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:28.681058 3882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:28.681304 3882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:28.714893 3882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "3123c47794c54cb0adfaca4b422109d3"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "1979e70afaf6efa926a8f7d1bc0c8db3"
server_key_iv: "ed6ef18d4f725252f528cc70d8482aee"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.715868 3882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "3123c47794c54cb0adfaca4b422109d3"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "1979e70afaf6efa926a8f7d1bc0c8db3"
server_key_iv: "ed6ef18d4f725252f528cc70d8482aee"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.722406 3882 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 02:29:28.745262 3902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.746920 3882 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.002s
I20260807 02:29:28.747130 3882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "3123c47794c54cb0adfaca4b422109d3"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "1979e70afaf6efa926a8f7d1bc0c8db3"
server_key_iv: "ed6ef18d4f725252f528cc70d8482aee"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:28.747295 3882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:28.773548 3882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:28.777885 3882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:28.778182 3882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:28.779067 3882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:28.780340 3882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:28.780460 3882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.780576 3882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:28.780658 3882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:28.795076 3882 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:37567
I20260807 02:29:28.796355 3882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:28.798084 4073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:37567 every 8 connection(s)
I20260807 02:29:28.803519 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 3882
I20260807 02:29:28.803645 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:29:28.803960 17620 external_mini_cluster.cc:1428] Setting key 3353cd20d0dcc5830c82ddfb9626a799
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:28.818269 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:28.802125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:42043 (local address 127.17.53.62:36431)
0807 02:29:28.802266 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:28.802270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:28.802831 (+ 561us) server_negotiation.cc:407] Connection header received
0807 02:29:28.804142 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:28.804146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:28.804205 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:28.804262 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:28.806237 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.807073 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.809005 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.809377 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.809456 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:28.813201 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:28.813206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:28.813209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:28.813242 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:28.815285 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:28.816032 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:28.816036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:28.816038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:28.816142 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:28.816605 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:28.816623 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:28.816625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:28.816920 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:28.817020 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:28.817921 (+ 901us) server_negotiation.cc:299] Negotiation successful
0807 02:29:28.818076 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:29:28.820019 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:28.800917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.2:42043)
0807 02:29:28.802642 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:28.802719 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:28.803872 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:28.804944 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:28.804953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:28.805347 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:28.806054 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:28.806079 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.807249 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.807256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:28.808856 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:28.808868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:28.809722 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:28.809727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:28.809958 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:28.810749 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:28.810771 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:28.812702 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:28.815451 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:28.815457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:28.815475 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:28.815887 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:28.816278 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:28.816281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:28.816284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:28.816474 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:28.817099 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:28.817106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:28.817572 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 02:29:28.818491 (+ 919us) client_negotiation.cc:240] Negotiation successful
0807 02:29:28.818869 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"thread_start_us":168,"threads_started":1}
I20260807 02:29:28.821523 4075 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:28.821802 4075 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:28.822436 4075 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:28.823735 3569 ts_manager.cc:194] Registered new tserver with Master: 3123c47794c54cb0adfaca4b422109d3 (127.17.53.2:37567)
I20260807 02:29:28.824527 3569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:42043
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:29:28.848706 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:28 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
W20260807 02:29:29.026016 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:29.026428 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:29.026496 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:29.032945 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:29.033044 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:29.033136 4080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:29:29.041467 4080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:29.043188 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:29.044638 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:29.055601 4090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.056180 4086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.056375 4080 server_base.cc:1034] running on GCE node
W20260807 02:29:29.056936 4088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.057412 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:29.058145 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:29.059332 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786069769059296 us; error 51 us; skew 500 ppm
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:29.062332 4080 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:29:29.063318 4080 webserver.cc:460] Webserver started at http://127.17.53.3:33219/ using document root <none> and password file <none>
I20260807 02:29:29.064107 4080 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:29.064245 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:29.064548 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:29.082861 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "5e76739655fb410998dd32e65e45c05b"
format_stamp: "Formatted at 2026-08-07 02:29:29 on dist-test-slave-g170"
server_key: "1b59dde077ebd65c758accacfd13feb0"
server_key_iv: "37371c53ab16956bc54817369bd556b8"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:29.083797 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "5e76739655fb410998dd32e65e45c05b"
format_stamp: "Formatted at 2026-08-07 02:29:29 on dist-test-slave-g170"
server_key: "1b59dde077ebd65c758accacfd13feb0"
server_key_iv: "37371c53ab16956bc54817369bd556b8"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:29.089988 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 02:29:29.114346 4100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.122678 4080 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260807 02:29:29.122975 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "5e76739655fb410998dd32e65e45c05b"
format_stamp: "Formatted at 2026-08-07 02:29:29 on dist-test-slave-g170"
server_key: "1b59dde077ebd65c758accacfd13feb0"
server_key_iv: "37371c53ab16956bc54817369bd556b8"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:29.123208 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:29.155407 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:29.159447 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:29.159693 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:29.160440 4080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:29.161499 4080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:29.161576 4080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.161695 4080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:29.161742 4080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.174535 4080 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38289
I20260807 02:29:29.174751 4269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38289 every 8 connection(s)
I20260807 02:29:29.175765 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:29:29.183180 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 4080
I20260807 02:29:29.183310 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:29:29.183610 17620 external_mini_cluster.cc:1428] Setting key 3173f7ca5dc1fc765fa0e686d739d49a
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:29.191287 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:29.178009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:35625 (local address 127.17.53.62:36431)
0807 02:29:29.178173 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:29.178178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:29.179124 (+ 946us) server_negotiation.cc:407] Connection header received
0807 02:29:29.180245 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:29.180249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:29.180319 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:29.180437 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:29.181736 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.182259 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.183772 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.184101 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.184181 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:29.187387 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:29.187393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:29.187401 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:29.187426 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:29.189042 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:29.189610 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:29.189614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:29.189615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:29.189717 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:29.190083 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:29.190088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:29.190090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:29.190289 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:29.190405 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:29.190980 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 02:29:29.191115 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:29:29.192261 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:29.178417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.3:35625)
0807 02:29:29.178954 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:29.179010 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:29.179952 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:29.180635 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:29.180642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:29.181024 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:29.181552 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:29.181566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.182388 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.182397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:29.183611 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:29.183625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.184598 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.184602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:29.184907 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:29.185735 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:29.185752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:29.187219 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:29.189196 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:29.189202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:29.189226 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:29.189495 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:29.189814 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:29.189826 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:29.189828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:29.189981 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:29.190455 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:29.190461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:29.190725 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 02:29:29.190929 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 02:29:29.191270 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":166,"threads_started":1}
I20260807 02:29:29.193852 4270 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:29.194118 4270 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:29.194672 4270 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:29.195808 3569 ts_manager.cc:194] Registered new tserver with Master: 5e76739655fb410998dd32e65e45c05b (127.17.53.3:38289)
I20260807 02:29:29.196386 3569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:35625
I20260807 02:29:29.198201 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:29:29.199131 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 3685
I20260807 02:29:29.222184 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 3882
I20260807 02:29:29.236311 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 4080
I20260807 02:29:29.251503 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 3521
I20260807 02:29:29.272238 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36431
--webserver_interface=127.17.53.62
--webserver_port=37221
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36431
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:29.385635 4274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:29.385898 4274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:29.385938 4274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:29.390026 4274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:29.390084 4274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:29.390105 4274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:29.390121 4274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:29.390159 4274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:29.395589 4274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36431
--ranger_config_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=37221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:29.396644 4274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:29.397686 4274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:29.406255 4284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.406272 4287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.406385 4274 server_base.cc:1034] running on GCE node
W20260807 02:29:29.406265 4290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.407275 4274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:29.408461 4274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:29.409689 4274 hybrid_clock.cc:648] HybridClock initialized: now 1786069769409657 us; error 50 us; skew 500 ppm
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:29.412950 4274 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:29.414112 4274 webserver.cc:460] Webserver started at http://127.17.53.62:37221/ using document root <none> and password file <none>
I20260807 02:29:29.414732 4274 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:29.414812 4274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:29.499387 4274 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:29:29.501870 4300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.503505 4274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 02:29:29.503660 4274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "597b0ac1d41940bdb41ac9a9d0edd736"
format_stamp: "Formatted at 2026-08-07 02:29:27 on dist-test-slave-g170"
server_key: "6b7bb147f064099b582431a7553b6f8d"
server_key_iv: "fff8bc1b13da5bf748119446474b59ce"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:29.504452 4274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:29.525714 4274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:29.529122 4274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:29.529590 4274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:29.536798 4274 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36431
I20260807 02:29:29.536827 4370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36431 every 8 connection(s)
I20260807 02:29:29.537909 4274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:29.541425 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 4274
I20260807 02:29:29.541965 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:45165
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:36431
--webserver_port=46561
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:29.543226 4371 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:29:29.545603 4371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Bootstrap starting.
I20260807 02:29:29.564152 4371 log.cc:824] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:29.571398 4371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:29:29.571866 4371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Bootstrap complete.
I20260807 02:29:29.576550 4371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:29.577215 4371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597b0ac1d41940bdb41ac9a9d0edd736, State: Initialized, Role: FOLLOWER
I20260807 02:29:29.577729 4371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:29.577881 4371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:29.577970 4371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:29.578094 4371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:29:29.580144 4371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:29.580497 4371 leader_election.cc:302] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597b0ac1d41940bdb41ac9a9d0edd736; no voters:
I20260807 02:29:29.580895 4371 leader_election.cc:288] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 02:29:29.580983 4376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 02:29:29.581234 4376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [term 2 LEADER]: Becoming Leader. State: Replica: 597b0ac1d41940bdb41ac9a9d0edd736, State: Running, Role: LEADER
I20260807 02:29:29.581620 4376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } }
I20260807 02:29:29.582306 4371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:29.582609 4376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597b0ac1d41940bdb41ac9a9d0edd736. Latest consensus state: current_term: 2 leader_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } } }
I20260807 02:29:29.582738 4376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:29.583428 4377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597b0ac1d41940bdb41ac9a9d0edd736" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36431 } } }
I20260807 02:29:29.583595 4377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:29.583937 4381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:29.587432 4381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:29.588011 4371 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 02:29:29.588230 4381 catalog_manager.cc:1259] Loaded cluster ID: 34f4c009743e4e63a12f3a2c378a5759
I20260807 02:29:29.588342 4381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:29.590816 4381 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:29.591499 4381 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Loaded TSK: 0
I20260807 02:29:29.592736 4381 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:29:29.689316 4373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:29.689594 4373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:29.689695 4373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:29.694355 4373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:29.694445 4373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:29.694553 4373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:29.700379 4373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:45165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=46561
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:29.701802 4373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:29.703369 4373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:29.719578 4434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.719844 4431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.720170 4428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.721091 4373 server_base.cc:1034] running on GCE node
I20260807 02:29:29.721618 4373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:29.722548 4373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:29.723775 4373 hybrid_clock.cc:648] HybridClock initialized: now 1786069769723744 us; error 47 us; skew 500 ppm
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:29.727600 4373 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:29.728940 4373 webserver.cc:460] Webserver started at http://127.17.53.1:46561/ using document root <none> and password file <none>
I20260807 02:29:29.729736 4373 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:29.729862 4373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:29.757417 4373 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:29:29.761070 4444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.762948 4373 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:29:29.763100 4373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "18a5a3f9a7b0400ca91c51c6b1667586"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "3597e11fc67ffad2b4b3c3591ba88b1e"
server_key_iv: "b495d6b6ccecc0a73ba5908398f82995"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:29.763811 4373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:29.780560 4373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:29.783936 4373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:29.784179 4373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:29.784982 4373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:29.785936 4373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:29.785985 4373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.786027 4373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:29.786067 4373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:29.797534 4373 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45165
I20260807 02:29:29.797616 4601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45165 every 8 connection(s)
I20260807 02:29:29.799062 4373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:29.803261 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 4373
I20260807 02:29:29.803768 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:37567
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:36431
--webserver_port=41167
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:29.835719 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:29.802713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.1:42615)
0807 02:29:29.805275 (+ 2562us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:29.805342 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:29.806794 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:29.809727 (+ 2933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:29.809739 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:29.810160 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:29.810879 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:29.810902 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.812952 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.812956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:29.814453 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:29.814465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.815826 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.815833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:29.816031 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:29.816834 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:29.816857 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:29.818569 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:29.821311 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:29.821318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:29.821333 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:29.821686 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:29.822135 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:29.822139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:29.822141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:29.822368 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:29.823238 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:29.823245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:29.823575 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 02:29:29.823817 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 02:29:29.834502 (+ 10685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2306,"thread_start_us":176,"threads_started":1}
I20260807 02:29:29.837189 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:29.804284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:42615 (local address 127.17.53.62:36431)
0807 02:29:29.807733 (+ 3449us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:29.807772 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:29.807829 (+ 57us) server_negotiation.cc:407] Connection header received
0807 02:29:29.808182 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:29.808198 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:29.808676 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:29.809140 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:29.811118 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.812779 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.814606 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:29.815367 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:29.815545 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:29.818829 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:29.818836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:29.818856 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:29.818890 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:29.821169 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:29.821853 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:29.821860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:29.821866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:29.822013 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:29.822490 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:29.822494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:29.822495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:29.823038 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:29.823166 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:29.824054 (+ 888us) server_negotiation.cc:299] Negotiation successful
0807 02:29:29.836217 (+ 12163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":125,"threads_started":1}
I20260807 02:29:29.841331 4602 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:29.841599 4602 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:29.842237 4602 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:29.844743 4602 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36431 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 02:29:29.844905 4602 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:29.845082 4602 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:29.846123 4602 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:29.846314 4602 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:29.847141 4602 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:29:29.958251 4607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:29.958683 4607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:29.958755 4607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:29.964582 4607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:29.964655 4607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:29.964720 4607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:29.971334 4607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:37567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=41167
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:29.972497 4607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:29.973841 4607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:29.984827 4623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.984723 4619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:29.985049 4617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:29.985395 4607 server_base.cc:1034] running on GCE node
I20260807 02:29:29.988735 4607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:29.989535 4607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:29.990715 4607 hybrid_clock.cc:648] HybridClock initialized: now 1786069769990686 us; error 48 us; skew 500 ppm
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:29.993912 4607 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:29.994935 4607 webserver.cc:460] Webserver started at http://127.17.53.2:41167/ using document root <none> and password file <none>
I20260807 02:29:29.995537 4607 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:29.995651 4607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 02:29:29 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:30.020018 4607 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:29:30.023540 4633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.025347 4607 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 02:29:30.025480 4607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "3123c47794c54cb0adfaca4b422109d3"
format_stamp: "Formatted at 2026-08-07 02:29:28 on dist-test-slave-g170"
server_key: "1979e70afaf6efa926a8f7d1bc0c8db3"
server_key_iv: "ed6ef18d4f725252f528cc70d8482aee"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:30.026187 4607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:30.058283 4607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:30.062309 4607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:30.062630 4607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:30.063452 4607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:30.064602 4607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:30.064710 4607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.064851 4607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:30.064924 4607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.078802 4607 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:37567
I20260807 02:29:30.078863 4799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:37567 every 8 connection(s)
I20260807 02:29:30.080425 4607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:30.085184 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 4607
I20260807 02:29:30.086027 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.53.20:54987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:38289
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:36431
--webserver_port=33219
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:30 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069769, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:30.098695 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:30.082763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:48329 (local address 127.17.53.62:36431)
0807 02:29:30.082921 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:30.082926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:30.084299 (+ 1373us) server_negotiation.cc:407] Connection header received
0807 02:29:30.085726 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:30.085730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:30.085815 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:30.085876 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:30.087674 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.088526 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.090432 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.090809 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.090894 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:30.094234 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:30.094239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:30.094242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:30.094273 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:30.096134 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.096812 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.096816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.096818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.096910 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.097318 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.097322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.097324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.097544 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:30.097630 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:30.098346 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 02:29:30.098502 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:29:30.100353 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:30.083333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.2:48329)
0807 02:29:30.084123 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:30.084187 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:30.085435 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:30.086319 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:30.086328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:30.086689 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:30.087486 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:30.087509 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.088696 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.088700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:30.090280 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:30.090293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.091115 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.091119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:30.091404 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:30.092130 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:30.092153 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:30.094028 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:30.096276 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:30.096282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:30.096299 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:30.096641 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:30.097024 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:30.097027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:30.097029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:30.097200 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:30.097701 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:30.097708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:30.098046 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 02:29:30.098952 (+ 906us) client_negotiation.cc:240] Negotiation successful
0807 02:29:30.099292 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":147,"threads_started":1}
I20260807 02:29:30.101780 4800 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:30.102062 4800 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.102725 4800 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:30.104526 4800 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36431 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 02:29:30.104657 4800 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.104869 4800 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:30.105680 4800 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.105861 4800 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:30.106601 4800 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:29:30.269544 4803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:30.270099 4803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:30.270267 4803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:30.277457 4803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:30.277621 4803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:30.277781 4803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:29:30.286935 4803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.53.20:54987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:38289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=33219
--tserver_master_addrs=127.17.53.62:36431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:30.288375 4803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:30.289611 4803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:30.302863 4818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:30.302855 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:30.302855 4815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:30.304544 4803 server_base.cc:1034] running on GCE node
I20260807 02:29:30.305059 4803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:30.305788 4803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:30.306972 4803 hybrid_clock.cc:648] HybridClock initialized: now 1786069770306942 us; error 51 us; skew 500 ppm
Aug 07 02:29:30 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:30.310243 4803 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:29:30.311254 4803 webserver.cc:460] Webserver started at http://127.17.53.3:33219/ using document root <none> and password file <none>
I20260807 02:29:30.311837 4803 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:30.311913 4803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 02:29:30 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:30.334120 4803 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260807 02:29:30.336838 4826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.338347 4803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:30.338500 4803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "5e76739655fb410998dd32e65e45c05b"
format_stamp: "Formatted at 2026-08-07 02:29:29 on dist-test-slave-g170"
server_key: "1b59dde077ebd65c758accacfd13feb0"
server_key_iv: "37371c53ab16956bc54817369bd556b8"
server_key_version: "kuduclusterkey@0"
I20260807 02:29:30.339162 4803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:30.353942 4803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:30.357307 4803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:30.357556 4803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:30.358223 4803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:30.359135 4803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:30.359218 4803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.359338 4803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:30.359385 4803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:30.373226 4803 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:38289
I20260807 02:29:30.373592 5004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:38289 every 8 connection(s)
I20260807 02:29:30.374747 4803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:29:30.382562 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 4803
Aug 07 02:29:30 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069770, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:30.395526 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:30.377162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:56855 (local address 127.17.53.62:36431)
0807 02:29:30.377335 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:30.377340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:30.378380 (+ 1040us) server_negotiation.cc:407] Connection header received
0807 02:29:30.379834 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:30.379838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:30.379886 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:30.379963 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:30.381599 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.382424 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.384712 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.385075 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.385160 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:30.388140 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:30.388145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:30.388147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:30.388173 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:30.392792 (+ 4619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.393477 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.393481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.393484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.393579 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.394087 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.394091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.394093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.394326 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:30.394405 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:30.395187 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 02:29:30.395335 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:29:30.396545 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:30.377567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36431 (local address 127.17.53.3:56855)
0807 02:29:30.378185 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:30.378249 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:30.379561 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:30.380193 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:30.380202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:30.380633 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:30.381428 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:30.381452 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.382615 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.382619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:30.384564 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:30.384577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.385377 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.385381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:30.385569 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:30.386237 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:30.386258 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:30.387976 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:30.392939 (+ 4963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:30.392946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:30.392963 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:30.393348 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:30.393734 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:30.393737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:30.393739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:30.393932 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:30.394497 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:30.394504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:30.394862 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 02:29:30.395098 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 02:29:30.395464 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":174,"threads_started":1}
I20260807 02:29:30.397878 5005 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36431
I20260807 02:29:30.398135 5005 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.398732 5005 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:30.401021 5005 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:36431 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 02:29:30.401165 5005 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.401705 5005 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:30.403060 5005 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.403247 5005 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
W20260807 02:29:30.404161 5005 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 02:29:30.404690 5013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:30.383345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.17.53.62:36431)
0807 02:29:30.384425 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:30.384437 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:30.387650 (+ 3213us) server_negotiation.cc:407] Connection header received
0807 02:29:30.387697 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:30.387700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:30.387751 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:30.387848 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:30.392882 (+ 5034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.393972 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.398431 (+ 4459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:30.398829 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:30.398965 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:30.400958 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:30.400962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:30.400964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:30.400989 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:30.402700 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.403427 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.403430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.403431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.403518 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:30.403858 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:30.403861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:30.403863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:30.404035 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:30.404268 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:30.404471 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 02:29:30.404571 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260807 02:29:30.847543 4602 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:30.848026 4602 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:30.849903 4320 ts_manager.cc:194] Registered new tserver with Master: 18a5a3f9a7b0400ca91c51c6b1667586 (127.17.53.1:45165)
I20260807 02:29:30.852321 4320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:42615
Aug 07 02:29:31 dist-test-slave-g170 krb5kdc[2889](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786069771, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:31.106853 4800 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:31.107162 4800 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:31.108001 4320 ts_manager.cc:194] Registered new tserver with Master: 3123c47794c54cb0adfaca4b422109d3 (127.17.53.2:37567)
I20260807 02:29:31.108985 4320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:48329
I20260807 02:29:31.404450 5005 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:31.404779 5005 heartbeater.cc:511] Master 127.17.53.62:36431 requested a full tablet report, sending...
I20260807 02:29:31.405686 4320 ts_manager.cc:194] Registered new tserver with Master: 5e76739655fb410998dd32e65e45c05b (127.17.53.3:38289)
I20260807 02:29:31.406499 4320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:56855
I20260807 02:29:31.410826 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:29:31.427029 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.412489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.17.53.62:36431)
0807 02:29:31.414141 (+ 1652us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.414145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.414160 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:29:31.414280 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.414286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.414332 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.414445 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:31.415131 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.415664 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.416865 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.417114 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.417225 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.418683 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:31.418687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:31.418690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:31.418716 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:31.420371 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:31.421110 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:31.421114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:31.421116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:31.421214 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:31.421535 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:31.421538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:31.421540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:31.421724 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:31.421866 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.425727 (+ 3861us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.425874 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1565,"thread_start_us":79,"threads_started":1}
I20260807 02:29:31.431602 4320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:29:31.435920 4320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 02:29:31 dist-test-slave-g170 krb5kdc[2889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069771, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for HTTP/127.17.53.20@KRBTEST.COM
I20260807 02:29:31.459880 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.448254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45165 (local address 127.0.0.1:46644)
0807 02:29:31.451518 (+ 3264us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.451572 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:31.451710 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:31.454909 (+ 3199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:31.454916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:31.454944 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:31.455084 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.455090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.456677 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.456681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.458152 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.458164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.459328 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.459332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.459449 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.459496 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 02:29:31.459585 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 02:29:31.459701 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3042,"thread_start_us":101,"threads_started":1}
I20260807 02:29:31.460543 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.449146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.17.53.1:45165)
0807 02:29:31.454375 (+ 5229us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.454382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.454412 (+ 30us) server_negotiation.cc:407] Connection header received
0807 02:29:31.454497 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.454503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.454727 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.454897 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:31.455237 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.456536 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.458322 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.459180 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.460095 (+ 915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.460259 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.460314 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.460405 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5102,"thread_start_us":76,"threads_started":1}
I20260807 02:29:31.463951 4510 tablet_service.cc:1467] Processing CreateTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd (DEFAULT_TABLE table=test-table [id=e5f8dba080c2461d83c481b647d96cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:31.465561 4510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9f577cef8e444db2ebc174b6c26cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:31.465776 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.451066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:37567 (local address 127.0.0.1:47238)
0807 02:29:31.451998 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.452010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:31.452080 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:31.461372 (+ 9292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:31.461376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:31.461395 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:31.461481 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.461487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.462969 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.462972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.464159 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.464170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.465386 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.465390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.465487 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.465514 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:29:31.465562 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 02:29:31.465628 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":112,"threads_started":1}
I20260807 02:29:31.466432 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.451254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.17.53.2:37567)
0807 02:29:31.460983 (+ 9729us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.460989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.461010 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:29:31.461075 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.461079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.461225 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.461400 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:31.461617 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.462826 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.464327 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.465238 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.465981 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.466136 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.466188 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.466279 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9625,"thread_start_us":112,"threads_started":1}
I20260807 02:29:31.471235 4683 tablet_service.cc:1467] Processing CreateTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd (DEFAULT_TABLE table=test-table [id=e5f8dba080c2461d83c481b647d96cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:31.472412 4683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9f577cef8e444db2ebc174b6c26cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:31.474848 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.451769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38289 (local address 127.0.0.1:35074)
0807 02:29:31.454076 (+ 2307us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.454091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:31.455133 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:31.455858 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:31.455861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:31.455880 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:31.455981 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.455987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.471610 (+ 15623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.471613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.473058 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.473071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.474426 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.474430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.474522 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.474548 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:29:31.474621 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 02:29:31.474693 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2248,"thread_start_us":93,"threads_started":1}
I20260807 02:29:31.475302 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.452175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35074 (local address 127.17.53.3:38289)
0807 02:29:31.452536 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.452543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.455269 (+ 2726us) server_negotiation.cc:407] Connection header received
0807 02:29:31.455357 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.455363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.455557 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.455697 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:31.456117 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.471438 (+ 15321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.473231 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.474085 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.474204 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.474356 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.475041 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.475165 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
I20260807 02:29:31.478298 4888 tablet_service.cc:1467] Processing CreateTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd (DEFAULT_TABLE table=test-table [id=e5f8dba080c2461d83c481b647d96cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:31.479364 4888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9f577cef8e444db2ebc174b6c26cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:31.488209 5064 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Bootstrap starting.
I20260807 02:29:31.482244 5062 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Bootstrap starting.
I20260807 02:29:31.491029 5064 tablet_bootstrap.cc:654] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:31.491216 5062 tablet_bootstrap.cc:654] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:31.492152 5064 log.cc:824] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:31.492166 5062 log.cc:824] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:31.493003 5063 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Bootstrap starting.
I20260807 02:29:31.495496 5063 tablet_bootstrap.cc:654] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:31.495719 5062 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: No bootstrap required, opened a new log
I20260807 02:29:31.496002 5062 ts_tablet_manager.cc:1397] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260807 02:29:31.496405 5063 log.cc:824] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:31.499331 5063 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: No bootstrap required, opened a new log
I20260807 02:29:31.499552 5063 ts_tablet_manager.cc:1397] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 02:29:31.500734 5062 raft_consensus.cc:348] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.501073 5062 raft_consensus.cc:374] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:31.501134 5062 raft_consensus.cc:729] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a5a3f9a7b0400ca91c51c6b1667586, State: Initialized, Role: FOLLOWER
I20260807 02:29:31.501643 5062 consensus_queue.cc:260] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.502836 5062 ts_tablet_manager.cc:1428] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260807 02:29:31.503729 5063 raft_consensus.cc:348] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.504009 5063 raft_consensus.cc:374] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:31.504057 5063 raft_consensus.cc:729] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3123c47794c54cb0adfaca4b422109d3, State: Initialized, Role: FOLLOWER
I20260807 02:29:31.504879 5063 consensus_queue.cc:260] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.505569 5064 tablet_bootstrap.cc:492] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: No bootstrap required, opened a new log
I20260807 02:29:31.505879 5064 ts_tablet_manager.cc:1397] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260807 02:29:31.506186 5063 ts_tablet_manager.cc:1428] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 02:29:31.506626 4800 heartbeater.cc:503] Master 127.17.53.62:36431 was elected leader, sending a full tablet report...
I20260807 02:29:31.510246 5064 raft_consensus.cc:348] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.510552 5064 raft_consensus.cc:374] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:31.510613 5064 raft_consensus.cc:729] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e76739655fb410998dd32e65e45c05b, State: Initialized, Role: FOLLOWER
I20260807 02:29:31.511217 5064 consensus_queue.cc:260] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.512343 5064 ts_tablet_manager.cc:1428] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:29:31.512754 5005 heartbeater.cc:503] Master 127.17.53.62:36431 was elected leader, sending a full tablet report...
I20260807 02:29:31.515324 4602 heartbeater.cc:503] Master 127.17.53.62:36431 was elected leader, sending a full tablet report...
I20260807 02:29:31.516294 5076 raft_consensus.cc:482] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:31.516513 5076 raft_consensus.cc:504] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.518883 5076 leader_election.cc:288] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3123c47794c54cb0adfaca4b422109d3 (127.17.53.2:37567), 5e76739655fb410998dd32e65e45c05b (127.17.53.3:38289)
I20260807 02:29:31.524339 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.518638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:45353 (local address 127.17.53.2:37567)
0807 02:29:31.518778 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.518783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.519759 (+ 976us) server_negotiation.cc:407] Connection header received
0807 02:29:31.519967 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.519970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.520040 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.520088 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:31.520550 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.521398 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.523012 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.523602 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.523684 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.523752 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.524138 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.524216 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:29:31.525056 5083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.518546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:37567 (local address 127.17.53.1:45353)
0807 02:29:31.519675 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.519688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:31.519848 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:31.520260 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:31.520263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:31.520282 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:31.520423 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.520429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.521525 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.521529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.522864 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.522876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.523873 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.523877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.523986 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.524011 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 02:29:31.524697 (+ 686us) client_negotiation.cc:240] Negotiation successful
0807 02:29:31.524826 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":110,"threads_started":1}
I20260807 02:29:31.525918 4720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9f577cef8e444db2ebc174b6c26cdd" candidate_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3123c47794c54cb0adfaca4b422109d3" is_pre_election: true
I20260807 02:29:31.526260 4720 raft_consensus.cc:2454] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a5a3f9a7b0400ca91c51c6b1667586 in term 0.
I20260807 02:29:31.526875 4447 leader_election.cc:302] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a5a3f9a7b0400ca91c51c6b1667586, 3123c47794c54cb0adfaca4b422109d3; no voters:
I20260807 02:29:31.527261 5076 raft_consensus.cc:2781] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:29:31.527354 5076 raft_consensus.cc:482] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:29:31.527403 5076 raft_consensus.cc:3037] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:31.528249 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.519432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35473 (local address 127.17.53.3:38289)
0807 02:29:31.519536 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.519540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.524006 (+ 4466us) server_negotiation.cc:407] Connection header received
0807 02:29:31.524177 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.524180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.524224 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.524270 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:31.524647 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.525419 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.526997 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.527590 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.527672 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.527730 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.528063 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.528137 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 02:29:31.528628 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.519107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:38289 (local address 127.17.53.1:35473)
0807 02:29:31.523919 (+ 4812us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.523934 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:31.524068 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:31.524392 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:31.524394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:31.524410 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:31.524529 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.524534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.525566 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.525569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.526855 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:31.526867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.527827 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.527831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:31.527934 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.527955 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:29:31.528419 (+ 464us) client_negotiation.cc:240] Negotiation successful
0807 02:29:31.528494 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4744,"thread_start_us":100,"threads_started":1}
I20260807 02:29:31.529261 4940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9f577cef8e444db2ebc174b6c26cdd" candidate_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e76739655fb410998dd32e65e45c05b" is_pre_election: true
I20260807 02:29:31.529330 5076 raft_consensus.cc:504] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.529546 4940 raft_consensus.cc:2454] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a5a3f9a7b0400ca91c51c6b1667586 in term 0.
I20260807 02:29:31.529970 4720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9f577cef8e444db2ebc174b6c26cdd" candidate_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3123c47794c54cb0adfaca4b422109d3"
I20260807 02:29:31.530083 4720 raft_consensus.cc:3037] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:31.531467 4720 raft_consensus.cc:2454] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a5a3f9a7b0400ca91c51c6b1667586 in term 1.
I20260807 02:29:31.531852 4447 leader_election.cc:302] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a5a3f9a7b0400ca91c51c6b1667586, 3123c47794c54cb0adfaca4b422109d3; no voters:
I20260807 02:29:31.532052 5072 raft_consensus.cc:2781] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:31.532310 5076 leader_election.cc:288] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [CANDIDATE]: Term 1 election: Requested vote from peers 3123c47794c54cb0adfaca4b422109d3 (127.17.53.2:37567), 5e76739655fb410998dd32e65e45c05b (127.17.53.3:38289)
I20260807 02:29:31.532385 4940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9f577cef8e444db2ebc174b6c26cdd" candidate_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e76739655fb410998dd32e65e45c05b"
I20260807 02:29:31.532516 4940 raft_consensus.cc:3037] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:31.534138 4940 raft_consensus.cc:2454] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a5a3f9a7b0400ca91c51c6b1667586 in term 1.
I20260807 02:29:31.534483 5072 raft_consensus.cc:686] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [term 1 LEADER]: Becoming Leader. State: Replica: 18a5a3f9a7b0400ca91c51c6b1667586, State: Running, Role: LEADER
I20260807 02:29:31.534950 5072 consensus_queue.cc:237] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } }
I20260807 02:29:31.544899 4320 catalog_manager.cc:5495] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a5a3f9a7b0400ca91c51c6b1667586 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18a5a3f9a7b0400ca91c51c6b1667586" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 } health_report { overall_health: UNKNOWN } } }
W20260807 02:29:31.553716 4604 tablet.cc:2369] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:29:31.569787 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.564879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.17.53.1:45165)
0807 02:29:31.565005 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:31.565010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:31.565026 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:29:31.565113 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:31.565117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:31.565186 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:31.565313 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:29:31.565605 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.566466 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.567741 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:31.568107 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:31.568276 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:31.568685 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:29:31.569299 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:29:31.569435 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:31.569525 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 02:29:31.569595 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:29:31.577311 4720 raft_consensus.cc:1261] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 1 FOLLOWER]: Refusing update from remote peer 18a5a3f9a7b0400ca91c51c6b1667586: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:31.577307 4940 raft_consensus.cc:1261] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 1 FOLLOWER]: Refusing update from remote peer 18a5a3f9a7b0400ca91c51c6b1667586: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:31.578169 5084 consensus_queue.cc:1035] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3123c47794c54cb0adfaca4b422109d3" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 37567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:31.578687 5084 consensus_queue.cc:1035] T ab9f577cef8e444db2ebc174b6c26cdd P 18a5a3f9a7b0400ca91c51c6b1667586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e76739655fb410998dd32e65e45c05b" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 38289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 02:29:31.584662 4801 tablet.cc:2369] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:29:31.594163 5087 mvcc.cc:205] Tried to move back new op lower bound from 7315741784372129792 to 7315741784210751488. Current Snapshot: MvccSnapshot[applied={T|T < 7315741784372129792 or (T in {7315741784372129792})}]
I20260807 02:29:31.596868 5089 mvcc.cc:205] Tried to move back new op lower bound from 7315741784372129792 to 7315741784210751488. Current Snapshot: MvccSnapshot[applied={T|T < 7315741784372129792}]
I20260807 02:29:31.602429 4319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:31.602607 4319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:31.604666 5088 mvcc.cc:205] Tried to move back new op lower bound from 7315741784372129792 to 7315741784210751488. Current Snapshot: MvccSnapshot[applied={T|T < 7315741784372129792}]
I20260807 02:29:31.605300 4319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 597b0ac1d41940bdb41ac9a9d0edd736: Sending DeleteTablet for 3 replicas of tablet ab9f577cef8e444db2ebc174b6c26cdd
I20260807 02:29:31.606580 4510 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:31 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:46644
I20260807 02:29:31.606904 4683 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:31 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47238
I20260807 02:29:31.607096 4888 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9f577cef8e444db2ebc174b6c26cdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:31 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:35074
I20260807 02:29:31.607887 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 4373
I20260807 02:29:31.608647 5101 tablet_replica.cc:330] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: stopping tablet replica
I20260807 02:29:31.609000 5101 raft_consensus.cc:2229] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:31.609274 5101 raft_consensus.cc:2258] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:31.616704 5102 tablet_replica.cc:330] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: stopping tablet replica
I20260807 02:29:31.617238 5102 raft_consensus.cc:2229] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:31.617601 5102 raft_consensus.cc:2258] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:31.618908 5101 ts_tablet_manager.cc:1905] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:31.622114 5102 ts_tablet_manager.cc:1905] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:31.622251 5101 ts_tablet_manager.cc:1918] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:31.622426 5101 log.cc:1196] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/ab9f577cef8e444db2ebc174b6c26cdd
I20260807 02:29:31.622915 5101 ts_tablet_manager.cc:1939] T ab9f577cef8e444db2ebc174b6c26cdd P 3123c47794c54cb0adfaca4b422109d3: Deleting consensus metadata
I20260807 02:29:31.624529 4302 catalog_manager.cc:4841] TS 3123c47794c54cb0adfaca4b422109d3 (127.17.53.2:37567): tablet ab9f577cef8e444db2ebc174b6c26cdd (table test-table [id=e5f8dba080c2461d83c481b647d96cb9]) successfully deleted
I20260807 02:29:31.625183 5102 ts_tablet_manager.cc:1918] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:31.625313 5102 log.cc:1196] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/ab9f577cef8e444db2ebc174b6c26cdd
I20260807 02:29:31.625865 5102 ts_tablet_manager.cc:1939] T ab9f577cef8e444db2ebc174b6c26cdd P 5e76739655fb410998dd32e65e45c05b: Deleting consensus metadata
W20260807 02:29:31.625988 4303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:45165 (error 108)
I20260807 02:29:31.626925 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 4607
I20260807 02:29:31.631424 5049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:31.631064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45165 (local address 127.0.0.1:46652)
0807 02:29:31.631230 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:31.631322 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 02:29:31.631783 4303 catalog_manager.cc:4584] TS 18a5a3f9a7b0400ca91c51c6b1667586 (127.17.53.1:45165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab9f577cef8e444db2ebc174b6c26cdd: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45165: connect: Connection refused (error 111)
I20260807 02:29:31.636950 4301 catalog_manager.cc:4841] TS 5e76739655fb410998dd32e65e45c05b (127.17.53.3:38289): tablet ab9f577cef8e444db2ebc174b6c26cdd (table test-table [id=e5f8dba080c2461d83c481b647d96cb9]) successfully deleted
I20260807 02:29:31.642967 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 4803
I20260807 02:29:31.653934 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 4274
I20260807 02:29:31.670636 17620 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 02:29:32.017338 17620 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 02:29:32.017448 17620 mini_ranger.cc:66] Stopping Ranger...
I20260807 02:29:32.365411 17620 mini_ranger.cc:68] Stopped Ranger
2026-08-07 02:29:32.365 UTC [2933] LOG: received smart shutdown request
2026-08-07 02:29:32.371 UTC [2933] LOG: background worker "logical replication launcher" (PID 2943) exited with exit code 1
2026-08-07 02:29:32.371 UTC [2937] LOG: shutting down
2026-08-07 02:29:32.564 UTC [2933] LOG: database system is shut down
2026-08-07T02:29:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51556 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:29:32 dist-test-slave-g170 krb5kdc[5116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:29:32 dist-test-slave-g170 krb5kdc[5116](info): set up 2 sockets
Aug 07 02:29:32 dist-test-slave-g170 krb5kdc[5116](info): commencing operation
krb5kdc: starting...
W20260807 02:29:33.983880 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:29:34.123417 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36629
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:34.244513 5140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:34.244988 5140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:34.245054 5140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:34.249195 5140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:34.249258 5140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:34.249279 5140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:34.249295 5140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:34.249311 5140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:34.254808 5140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:34.255896 5140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:34.256964 5140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:34.266017 5156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:34.266026 5149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:34.266196 5140 server_base.cc:1034] running on GCE node
W20260807 02:29:34.266413 5152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:34.267051 5140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:34.268308 5140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:34.271593 5140 hybrid_clock.cc:648] HybridClock initialized: now 1786069774271564 us; error 57 us; skew 500 ppm
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:34.281133 5140 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:34.282269 5140 webserver.cc:460] Webserver started at http://127.17.53.62:34711/ using document root <none> and password file <none>
I20260807 02:29:34.282981 5140 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:34.283099 5140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:34.283397 5140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:34.285893 5140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "195717a6b8f14b3e88f171ad7732b6e9"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "97849c9269f0fe27e1c146d04c729f76"
server_key_iv: "cc43fa2b7f9480a5122453f4f4ff7119"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.286579 5140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "195717a6b8f14b3e88f171ad7732b6e9"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "97849c9269f0fe27e1c146d04c729f76"
server_key_iv: "cc43fa2b7f9480a5122453f4f4ff7119"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.291044 5140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:29:34.293912 5165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:34.295218 5140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:34.295399 5140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "195717a6b8f14b3e88f171ad7732b6e9"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "97849c9269f0fe27e1c146d04c729f76"
server_key_iv: "cc43fa2b7f9480a5122453f4f4ff7119"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.295531 5140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:34.322906 5140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:34.329648 5140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:34.329953 5140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:34.342126 5140 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36629
I20260807 02:29:34.342159 5238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36629 every 8 connection(s)
I20260807 02:29:34.343655 5140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:34.348016 5239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:34.352123 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5140
I20260807 02:29:34.352262 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:29:34.352523 17620 external_mini_cluster.cc:1428] Setting key bdaeb6b843dad40dcbeb6cfa6658b55c
I20260807 02:29:34.355518 5239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.53.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.61@KRBTEST.COM" created.
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:34.378249 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.358803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36629 (local address 127.0.0.1:44936)
0807 02:29:34.359399 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.359441 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.360228 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.361186 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.361192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.361641 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.361896 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.361909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.363038 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.363041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.363770 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.363780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.364418 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.364422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.364607 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.365318 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.365340 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.369747 (+ 4407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.374159 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.374174 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.374186 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.374539 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.375011 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.375018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.375021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.375210 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.376173 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.376182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.376373 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.376579 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.376989 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":55,"thread_start_us":171,"threads_started":1}
I20260807 02:29:34.378262 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.358804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.17.53.62:36629)
0807 02:29:34.360567 (+ 1763us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.360579 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.360600 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:29:34.360710 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.360714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.360997 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.361187 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.362044 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.362906 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.363890 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.364264 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.364488 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.369975 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.369987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.370007 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.370043 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.373956 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.374671 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.374680 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.374684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.374868 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.375338 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.375344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.375346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.375998 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.376197 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.376966 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.377208 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1653,"spinlock_wait_cycles":2304,"thread_start_us":213,"threads_started":1}
I20260807 02:29:34.381951 5239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:29:34.382930 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.382604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40180)
0807 02:29:34.382748 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.382836 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 02:29:34.383272 5166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
I20260807 02:29:34.384464 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.384135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40196)
0807 02:29:34.384255 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.384397 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 02:29:34.384626 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:45589: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
I20260807 02:29:34.388427 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:45589
--webserver_interface=127.17.53.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:34.433918 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } attempt: 1
I20260807 02:29:34.434876 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.434546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40212)
0807 02:29:34.434719 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.434793 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 02:29:34.435667 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.435433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40226)
0807 02:29:34.435552 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.435605 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 02:29:34.435829 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:45589: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
I20260807 02:29:34.503998 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } attempt: 2
I20260807 02:29:34.504901 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.504550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40232)
0807 02:29:34.504708 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.504822 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 02:29:34.505646 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.505423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40236)
0807 02:29:34.505534 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.505588 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 02:29:34.505786 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:45589: Network error: Client connection negotiation failed: client connection to 127.17.53.61:45589: connect: Connection refused (error 111)
W20260807 02:29:34.507258 5250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:34.507524 5250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:34.507634 5250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:34.511775 5250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:34.511859 5250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:34.511900 5250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:34.511957 5250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:34.511998 5250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:34.517445 5250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:45589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.53.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:34.518515 5250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:34.519573 5250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:34.528381 5259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:34.528395 5260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:34.528381 5262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:34.528816 5250 server_base.cc:1034] running on GCE node
I20260807 02:29:34.529484 5250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:34.530519 5250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:34.531764 5250 hybrid_clock.cc:648] HybridClock initialized: now 1786069774531727 us; error 36 us; skew 500 ppm
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:34.534683 5250 init.cc:377] Logged in from keytab as kudu/127.17.53.61@KRBTEST.COM (short username kudu)
I20260807 02:29:34.535597 5250 webserver.cc:460] Webserver started at http://127.17.53.61:41559/ using document root <none> and password file <none>
I20260807 02:29:34.536176 5250 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:34.536268 5250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:34.536501 5250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:34.538673 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data/instance:
uuid: "782cbbcbc39a4d22ad2df1fcee41a9da"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "1b7be48c3b23282ebe24fb943f2d6833"
server_key_iv: "19a14e1403f1ee659bc465162d887b90"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.539250 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal/instance:
uuid: "782cbbcbc39a4d22ad2df1fcee41a9da"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "1b7be48c3b23282ebe24fb943f2d6833"
server_key_iv: "19a14e1403f1ee659bc465162d887b90"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.543190 5250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:29:34.545974 5277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:34.547263 5250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:34.547425 5250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "782cbbcbc39a4d22ad2df1fcee41a9da"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "1b7be48c3b23282ebe24fb943f2d6833"
server_key_iv: "19a14e1403f1ee659bc465162d887b90"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.547540 5250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:34.563676 5250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:34.566951 5250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:34.567169 5250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:34.574427 5250 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.61:45589
I20260807 02:29:34.574443 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.61:45589 every 8 connection(s)
I20260807 02:29:34.575402 5250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
I20260807 02:29:34.575672 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5250
I20260807 02:29:34.575770 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-1/wal/instance
I20260807 02:29:34.576159 17620 external_mini_cluster.cc:1428] Setting key 3151cea611090204940ed1be15074219
I20260807 02:29:34.578310 5368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:34.582710 5368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.53.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.60@KRBTEST.COM" created.
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:34.596935 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } attempt: 3
I20260807 02:29:34.598232 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.585069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.17.53.62:36629)
0807 02:29:34.585199 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.585203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.586358 (+ 1155us) server_negotiation.cc:407] Connection header received
0807 02:29:34.587251 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.587254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.587316 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.587423 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.588558 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.589205 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.590429 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.590702 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.590824 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.593706 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.593712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.593716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.593766 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.595353 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.596038 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.596040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.596042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.596118 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.596547 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.596550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.596551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.596718 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.596996 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.597794 (+ 798us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.597971 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:29:34.600047 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.585669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36629 (local address 127.0.0.1:44952)
0807 02:29:34.586198 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.586238 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.587042 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.587589 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.587596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.587902 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.588411 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.588426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.589349 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.589352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.590214 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.590224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.590863 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.590867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.591082 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.591756 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.591778 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.593527 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.595479 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.595488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.595505 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.595924 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.596216 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.596222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.596225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.596440 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.597015 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.597025 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.597374 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.598510 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.598887 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":169,"threads_started":1}
I20260807 02:29:34.602048 5368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:29:34.608129 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.597368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40246)
0807 02:29:34.597571 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.597589 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.597664 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.598306 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.598309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.598553 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.598672 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.598681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.599732 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.599738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.600442 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.600456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.601308 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.601311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.601423 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.601899 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.601913 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.604003 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.606317 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.606323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.606326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.606597 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.606924 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.606926 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.606927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.607006 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.607521 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.607524 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.607697 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.607828 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.607963 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 02:29:34.608881 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.597454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.17.53.61:45589)
0807 02:29:34.597935 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.597943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.597961 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:29:34.597994 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.597997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.598152 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.598310 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.598792 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.599620 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.600602 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.601181 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.601310 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.604144 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.604154 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.604166 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.604191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.606177 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.606718 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.606726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.606728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.606813 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.607112 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.607117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.607122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.607407 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.607554 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.607920 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.608053 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":108,"threads_started":1}
I20260807 02:29:34.610670 5239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
I20260807 02:29:34.611590 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.611204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52796)
0807 02:29:34.611451 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.611513 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 02:29:34.611756 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.602626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40254)
0807 02:29:34.602716 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.602727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.602792 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.603254 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.603257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.603421 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.603500 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.603504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.604557 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.604560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.605513 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.605527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.606032 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.606037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.606150 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.606567 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.606578 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.607971 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.610155 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.610160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.610166 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.610360 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.610727 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.610730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.610732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.610853 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.611250 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.611253 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.611340 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.611461 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.611599 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 02:29:34.611882 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.602742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.17.53.61:45589)
0807 02:29:34.603074 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.603081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.603092 (+ 11us) server_negotiation.cc:407] Connection header received
0807 02:29:34.603118 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.603120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.603150 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.603251 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.603610 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.604444 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.605647 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.605911 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.605999 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.608123 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.608127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.608129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.608159 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.610024 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.610512 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.610516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.610517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.610623 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.610941 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.610944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.610946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.611144 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.611255 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.611519 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.611716 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
I20260807 02:29:34.612263 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:46191
--webserver_interface=127.17.53.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:34.612659 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.612153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52810)
0807 02:29:34.612490 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.612549 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":96,"threads_started":1}
W20260807 02:29:34.612849 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
I20260807 02:29:34.612906 5368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:29:34.613569 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.613326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52812)
0807 02:29:34.613442 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.613518 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 02:29:34.613853 5282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
I20260807 02:29:34.614821 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.614573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52824)
0807 02:29:34.614692 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.614750 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 02:29:34.614982 5368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
I20260807 02:29:34.644058 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 1
I20260807 02:29:34.645136 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.644724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52828)
0807 02:29:34.644959 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.645033 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 02:29:34.646152 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.645880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52836)
0807 02:29:34.646028 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.646078 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 02:29:34.646308 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
I20260807 02:29:34.664237 5368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 1
I20260807 02:29:34.665088 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.664788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52838)
0807 02:29:34.664951 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.665011 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 02:29:34.665848 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.665613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52848)
0807 02:29:34.665741 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.665794 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 02:29:34.665983 5368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
I20260807 02:29:34.721505 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 2
I20260807 02:29:34.722448 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.722116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52860)
0807 02:29:34.722290 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.722361 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 02:29:34.723402 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.723121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52866)
0807 02:29:34.723276 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.723337 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 02:29:34.723578 5239 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
W20260807 02:29:34.732376 5383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:34.732695 5383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:34.732810 5383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 02:29:34.734179 5368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 2
I20260807 02:29:34.735097 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.734782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52874)
0807 02:29:34.734934 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.735016 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 02:29:34.735898 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.735655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52880)
0807 02:29:34.735773 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.735833 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 02:29:34.736043 5368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46191: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46191: connect: Connection refused (error 111)
W20260807 02:29:34.737293 5383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:34.737375 5383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:34.737442 5383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:34.737497 5383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:34.737536 5383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:34.743345 5383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36629,127.17.53.61:45589,127.17.53.60:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.53.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:34.744570 5383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:34.745781 5383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:34.755750 5395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:34.755730 5397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:34.755730 5393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:34.756060 5383 server_base.cc:1034] running on GCE node
I20260807 02:29:34.756834 5383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:34.758100 5383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:34.759344 5383 hybrid_clock.cc:648] HybridClock initialized: now 1786069774759304 us; error 56 us; skew 500 ppm
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:34.762385 5383 init.cc:377] Logged in from keytab as kudu/127.17.53.60@KRBTEST.COM (short username kudu)
I20260807 02:29:34.763370 5383 webserver.cc:460] Webserver started at http://127.17.53.60:46225/ using document root <none> and password file <none>
I20260807 02:29:34.763981 5383 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:34.764078 5383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:34.764328 5383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:34.766492 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data/instance:
uuid: "2cf365e0dd9b457d94e108230ab87f70"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "da6d4d45ee68839241639f9284f0f148"
server_key_iv: "e2687fbfece5017d90b556f4aba2d94f"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.767071 5383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal/instance:
uuid: "2cf365e0dd9b457d94e108230ab87f70"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "da6d4d45ee68839241639f9284f0f148"
server_key_iv: "e2687fbfece5017d90b556f4aba2d94f"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.770967 5383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 02:29:34.773515 5409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:34.774821 5383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:29:34.774976 5383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "2cf365e0dd9b457d94e108230ab87f70"
format_stamp: "Formatted at 2026-08-07 02:29:34 on dist-test-slave-g170"
server_key: "da6d4d45ee68839241639f9284f0f148"
server_key_iv: "e2687fbfece5017d90b556f4aba2d94f"
server_key_version: "encryptionkey@0"
I20260807 02:29:34.775099 5383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:34.796327 5383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:34.800175 5383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:34.800397 5383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:34.808015 5383 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.60:46191
I20260807 02:29:34.808045 5500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.60:46191 every 8 connection(s)
I20260807 02:29:34.809046 5383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
I20260807 02:29:34.810317 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5383
I20260807 02:29:34.810460 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786069602236550-17620-0/minicluster-data/master-2/wal/instance
I20260807 02:29:34.810765 17620 external_mini_cluster.cc:1428] Setting key f047676fc442a9b86b49b5b8aedadb62
I20260807 02:29:34.812451 5501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:34.816366 5501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:29:34.822762 5239 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 3
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:34.826700 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.812339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.17.53.60:46191)
0807 02:29:34.812909 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.812926 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.812965 (+ 39us) server_negotiation.cc:407] Connection header received
0807 02:29:34.813675 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.813693 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.814031 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.814381 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.815220 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.816245 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.817242 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.817755 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.817934 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.820837 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.820850 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.820860 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.820887 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.822747 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.823296 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.823303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.823309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.823460 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.823824 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.823828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.823832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.824391 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.824639 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.824937 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.825569 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":335,"thread_start_us":145,"threads_started":1}
I20260807 02:29:34.827394 5368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } attempt: 3
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:29:34.829643 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.818224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.17.53.62:36629)
0807 02:29:34.818414 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.818421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.818841 (+ 420us) server_negotiation.cc:407] Connection header received
0807 02:29:34.819006 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.819009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.819079 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.819181 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.820074 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.820645 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.821749 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.822101 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.822224 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.824813 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.824817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.824820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.824846 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.826596 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.827154 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.827158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.827160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.827248 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.827562 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.827565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.827567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.827806 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.827882 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.829306 (+ 1424us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.829443 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:29:34.830149 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.818138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36629 (local address 127.0.0.1:44964)
0807 02:29:34.818713 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.818755 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.818898 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.819252 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.819261 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.819710 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.819925 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.819936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.820830 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.820836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.821624 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.821635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.822229 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.822232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.822343 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.822853 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.822871 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.824599 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.826726 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.826735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.826741 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.827036 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.827343 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.827346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.827348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.827441 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.828847 (+ 1406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.828854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.829024 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.829240 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.829440 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":201,"threads_started":1}
I20260807 02:29:34.831503 5501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:29:34.833567 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.823186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52904)
0807 02:29:34.823393 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.823408 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.823488 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.823777 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.823780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.824007 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.824106 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.824112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.825140 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.825143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.825709 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.825723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.826283 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.826287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.826379 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.826807 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.826821 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.828275 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.831735 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.831738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.831740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.832014 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.832371 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.832376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.832378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.832481 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.833018 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.833021 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.833119 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.833230 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.833385 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:29:34.834086 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.823236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.17.53.60:46191)
0807 02:29:34.823567 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.823581 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.823593 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:29:34.823625 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.823633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.823664 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.823759 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.824258 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.824999 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.825882 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.826158 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.826279 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.828424 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.828428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.828430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.828454 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.831610 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.832142 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.832146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.832148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.832251 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.832607 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.832610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.832613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.832901 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.833025 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.833770 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.833902 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:29:34.837984 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.828100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52912)
0807 02:29:34.828228 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.828245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.828334 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.829112 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.829116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.829322 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.829442 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.829455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.830509 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.830514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.831286 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.831301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.831928 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.831932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.832014 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.832482 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.832496 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.834134 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.836185 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.836189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.836191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.836401 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.836837 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.836841 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.836843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.836949 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.837498 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.837501 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.837595 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.837689 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.837840 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:34.838506 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.828356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.17.53.60:46191)
0807 02:29:34.828831 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.828836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.828847 (+ 11us) server_negotiation.cc:407] Connection header received
0807 02:29:34.828873 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.828875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.828920 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.829068 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.829588 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.830373 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.831457 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.831777 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.832333 (+ 556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.834292 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.834295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.834297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.834316 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.836068 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.836506 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.836511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.836515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.836603 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.837077 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.837083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.837087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.837352 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.838170 (+ 818us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.838249 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.838378 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
I20260807 02:29:34.838598 5239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: Bootstrap starting.
I20260807 02:29:34.841809 5368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: Bootstrap starting.
I20260807 02:29:34.842703 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.831963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:45589 (local address 127.0.0.1:40256)
0807 02:29:34.832657 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.832671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.832740 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.833453 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.833456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.833621 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.833730 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.833736 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.834739 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.834742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.835502 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.835515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.836279 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.836282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.836364 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.836820 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.836834 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.838964 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.841024 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.841027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.841029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.841270 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.841616 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.841618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.841620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.841709 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.842239 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.842242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.842332 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.842439 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.842564 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622}
I20260807 02:29:34.843911 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.829910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44978 (local address 127.17.53.62:36629)
0807 02:29:34.830031 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.830035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.830815 (+ 780us) server_negotiation.cc:407] Connection header received
0807 02:29:34.830851 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.830854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.830894 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.830965 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.831870 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.832559 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.833963 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.834277 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.834358 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.839648 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.839650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.839652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.839673 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.841318 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.841755 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.841759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.841761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.841892 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.842221 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.842225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.842226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.842404 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.842718 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.843555 (+ 837us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.843757 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:29:34.844086 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.832521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.17.53.61:45589)
0807 02:29:34.833170 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.833174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.833187 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:29:34.833220 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.833223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.833262 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.833336 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.833877 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.834590 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.835651 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.835999 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.836105 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.839121 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.839124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.839127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.839152 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.840895 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.841402 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.841406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.841407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.841494 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.841818 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.841824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.841829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.842084 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.842160 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.843788 (+ 1628us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.843906 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565}
I20260807 02:29:34.844786 5501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:29:34.846939 5239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:34.847837 5239 log.cc:824] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:34.849125 5368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: Neither blocks nor log segments found. Creating new log.
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:29:34.850071 5368 log.cc:824] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:29:34 dist-test-slave-g170 krb5kdc[5116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:29:34.852550 5368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: No bootstrap required, opened a new log
I20260807 02:29:34.852561 5239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: No bootstrap required, opened a new log
I20260807 02:29:34.854501 5508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.845249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46191 (local address 127.0.0.1:52914)
0807 02:29:34.845400 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:34.845417 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:34.845497 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:34.845804 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:34.845808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:34.845979 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:34.846069 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.846076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.846924 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.846931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.847617 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:34.847627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.848192 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.848199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:34.848283 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.848755 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:34.848822 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:34.850406 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:34.852945 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.852948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.852950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.853179 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.853446 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:34.853449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:34.853450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:34.853532 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:34.853979 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:34.853982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:34.854074 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 02:29:34.854174 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 02:29:34.854315 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 02:29:34.854875 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.845175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40260 (local address 127.17.53.61:45589)
0807 02:29:34.845298 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.845303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.845365 (+ 62us) server_negotiation.cc:407] Connection header received
0807 02:29:34.845569 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.845572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.845616 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.845668 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.846548 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.847243 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.848218 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.848554 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.848613 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.851339 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.851343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.851345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.851369 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.852929 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.853444 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.853448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.853449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.853534 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.853863 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.853866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.853867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.854085 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.854231 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.854621 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.854735 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:29:34.855154 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.845504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.17.53.60:46191)
0807 02:29:34.845605 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.845609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.845621 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:29:34.845653 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.845658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.845692 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.845768 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.846205 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.846821 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.847749 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.848063 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.848189 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.850556 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.850560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.850563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.850584 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.852800 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.853280 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.853282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.853284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.853357 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.853623 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.853626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.853628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.853834 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.853948 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.854806 (+ 858us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.854938 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 02:29:34.857025 5239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.857298 5239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:34.857714 5239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195717a6b8f14b3e88f171ad7732b6e9, State: Initialized, Role: FOLLOWER
I20260807 02:29:34.858040 5368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.858323 5368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:34.858286 5239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.858448 5368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782cbbcbc39a4d22ad2df1fcee41a9da, State: Initialized, Role: FOLLOWER
I20260807 02:29:34.858539 5501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: Bootstrap starting.
I20260807 02:29:34.858646 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:29:34.859140 5368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.859834 5239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:34.860527 5368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:34.861438 5501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:34.862444 5501 log.cc:824] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:34.864576 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:34.865545 5501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: No bootstrap required, opened a new log
W20260807 02:29:34.866159 5555 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 02:29:34.864702 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: This master's current role is: FOLLOWER
W20260807 02:29:34.866372 5555 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:29:34.866400 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:34.866514 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 02:29:34.869747 5563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:29:34.869835 5563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:29:34.869812 5501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.870052 5501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:34.870117 5501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf365e0dd9b457d94e108230ab87f70, State: Initialized, Role: FOLLOWER
I20260807 02:29:34.870632 5501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.868160 5250 master_runner.cc:186] Error getting master registration for 127.17.53.60:46191: OK, instance_id { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" instance_seqno: 1786069774805830 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:29:34.871589 5250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:29:34.871999 5501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:34.871922 5140 master_runner.cc:186] Error getting master registration for 127.17.53.60:46191: OK, instance_id { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" instance_seqno: 1786069774805830 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:29:34.876221 5140 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:29:34.876832 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:34.876971 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:29:34.880429 5383 master_runner.cc:263] No leader master found from master 2cf365e0dd9b457d94e108230ab87f70
I20260807 02:29:34.880507 5383 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 02:29:34.883497 5590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:29:34.883582 5590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:29:34.886929 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.870288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.17.53.61:45589)
0807 02:29:34.870418 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.870423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.877554 (+ 7131us) server_negotiation.cc:407] Connection header received
0807 02:29:34.877694 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.877697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.877742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.877831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.878875 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.879536 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.880974 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.881297 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.881391 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.883315 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.883319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.883322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.883350 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.884832 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.885347 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.885351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.885353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.885453 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.885767 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.885771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.885772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.885999 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.886113 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.886537 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.886682 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 02:29:34.891661 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.871785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.17.53.62:36629)
0807 02:29:34.871894 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.871898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.874633 (+ 2735us) server_negotiation.cc:407] Connection header received
0807 02:29:34.874849 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.874853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.874904 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.874959 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.878628 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.879403 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.885350 (+ 5947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.885724 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.885824 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.887739 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.887743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.887748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.887780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.889593 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.890101 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.890106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.890108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.890209 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.890556 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.890562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.890564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.890835 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.890964 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.891317 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.891450 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:29:34.895215 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:34.871235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.17.53.60:46191)
0807 02:29:34.871346 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:34.871350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:34.876636 (+ 5286us) server_negotiation.cc:407] Connection header received
0807 02:29:34.877233 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:34.877236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:34.877297 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:34.877361 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:34.885315 (+ 7954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.886108 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.888879 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:34.889251 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:34.889334 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:34.891308 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:34.891312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:34.891315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:34.891340 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:34.893485 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.893910 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.893914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.893916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.894005 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:34.894313 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:34.894316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:34.894318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:34.894502 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:34.894657 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:34.894902 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 02:29:34.895024 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:29:34.994626 5515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:34.994854 5515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:34.995944 5515 leader_election.cc:288] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 195717a6b8f14b3e88f171ad7732b6e9 (127.17.53.62:36629), 2cf365e0dd9b457d94e108230ab87f70 (127.17.53.60:46191)
I20260807 02:29:34.996686 5201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195717a6b8f14b3e88f171ad7732b6e9" is_pre_election: true
I20260807 02:29:34.997069 5201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782cbbcbc39a4d22ad2df1fcee41a9da in term 0.
I20260807 02:29:34.997663 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf365e0dd9b457d94e108230ab87f70" is_pre_election: true
I20260807 02:29:34.997969 5465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782cbbcbc39a4d22ad2df1fcee41a9da in term 0.
I20260807 02:29:34.998468 5281 leader_election.cc:302] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 195717a6b8f14b3e88f171ad7732b6e9, 782cbbcbc39a4d22ad2df1fcee41a9da; no voters:
I20260807 02:29:34.999176 5515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:29:34.999285 5515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:29:34.999369 5515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:35.000896 5515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:35.005079 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cf365e0dd9b457d94e108230ab87f70"
I20260807 02:29:35.005244 5465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:35.006788 5465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782cbbcbc39a4d22ad2df1fcee41a9da in term 1.
I20260807 02:29:35.007174 5282 leader_election.cc:302] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cf365e0dd9b457d94e108230ab87f70, 782cbbcbc39a4d22ad2df1fcee41a9da; no voters:
I20260807 02:29:35.007589 5201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195717a6b8f14b3e88f171ad7732b6e9"
I20260807 02:29:35.007750 5201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:35.009142 5201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782cbbcbc39a4d22ad2df1fcee41a9da in term 1.
I20260807 02:29:35.009585 5515 leader_election.cc:288] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [CANDIDATE]: Term 1 election: Requested vote from peers 195717a6b8f14b3e88f171ad7732b6e9 (127.17.53.62:36629), 2cf365e0dd9b457d94e108230ab87f70 (127.17.53.60:46191)
I20260807 02:29:35.009784 5515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:35.009887 5515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [term 1 LEADER]: Becoming Leader. State: Replica: 782cbbcbc39a4d22ad2df1fcee41a9da, State: Running, Role: LEADER
I20260807 02:29:35.010265 5515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } }
I20260807 02:29:35.012409 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782cbbcbc39a4d22ad2df1fcee41a9da. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.012538 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:35.018071 5597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:35.019246 5597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:35.025527 5465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [term 1 FOLLOWER]: Refusing update from remote peer 782cbbcbc39a4d22ad2df1fcee41a9da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:35.026361 5593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:35.031122 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782cbbcbc39a4d22ad2df1fcee41a9da. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.031289 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:29:35.035353 5201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [term 1 FOLLOWER]: Refusing update from remote peer 782cbbcbc39a4d22ad2df1fcee41a9da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:35.036022 5593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [LEADER]: Connected to new peer: Peer: permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:35.040616 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782cbbcbc39a4d22ad2df1fcee41a9da. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.040786 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:29:35.047060 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.047199 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:35.048501 5513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.048614 5513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:29:35.049024 5599 mvcc.cc:205] Tried to move back new op lower bound from 7315741798495031296 to 7315741798445760512. Current Snapshot: MvccSnapshot[applied={T|T < 7315741798495031296}]
W20260807 02:29:35.052018 5422 tablet.cc:2369] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:29:35.053578 5593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.053711 5593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:35.054646 5571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195717a6b8f14b3e88f171ad7732b6e9" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36629 } } peers { permanent_uuid: "782cbbcbc39a4d22ad2df1fcee41a9da" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 45589 } } peers { permanent_uuid: "2cf365e0dd9b457d94e108230ab87f70" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46191 } } }
I20260807 02:29:35.054765 5571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:29:35.054971 5597 catalog_manager.cc:1347] Generated new cluster ID: 043da0a1686946029013ceccf3a88d7f
I20260807 02:29:35.055030 5597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:35.072252 5603 mvcc.cc:205] Tried to move back new op lower bound from 7315741798495031296 to 7315741798445760512. Current Snapshot: MvccSnapshot[applied={T|T < 7315741798495031296}]
I20260807 02:29:35.074995 5600 mvcc.cc:205] Tried to move back new op lower bound from 7315741798495031296 to 7315741798445760512. Current Snapshot: MvccSnapshot[applied={T|T < 7315741798495031296}]
I20260807 02:29:35.081700 5597 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:29:35.082998 5597 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:35.098134 5597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 782cbbcbc39a4d22ad2df1fcee41a9da: Generated new TSK 0
I20260807 02:29:35.099001 5597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:29:35.867874 5555 catalog_manager.cc:1259] Loaded cluster ID: 043da0a1686946029013ceccf3a88d7f
I20260807 02:29:35.867957 5555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: loading cluster ID for follower catalog manager: success
I20260807 02:29:35.869697 5555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: acquiring CA information for follower catalog manager: success
I20260807 02:29:35.871093 5555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 195717a6b8f14b3e88f171ad7732b6e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:29:35.884938 5590 catalog_manager.cc:1259] Loaded cluster ID: 043da0a1686946029013ceccf3a88d7f
I20260807 02:29:35.885008 5590 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: loading cluster ID for follower catalog manager: success
I20260807 02:29:35.886649 5590 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: acquiring CA information for follower catalog manager: success
I20260807 02:29:35.887785 5590 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2cf365e0dd9b457d94e108230ab87f70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:29:36.159756 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5140
I20260807 02:29:36.171281 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5250
I20260807 02:29:36.183151 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5383
2026-08-07T02:29:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3503 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:29:36 dist-test-slave-g170 krb5kdc[5617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:29:36 dist-test-slave-g170 krb5kdc[5617](info): set up 2 sockets
Aug 07 02:29:36 dist-test-slave-g170 krb5kdc[5617](info): commencing operation
krb5kdc: starting...
W20260807 02:29:37.528496 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:29:37 dist-test-slave-g170 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:29:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:29:37.660240 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35437
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:37.780233 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:37.780494 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:37.780534 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:37.784394 5639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:37.784446 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:37.784466 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:37.784482 5639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:37.784497 5639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:37.789711 5639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:37.790652 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:37.791621 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:37.800797 5650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:37.800812 5652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:37.800936 5639 server_base.cc:1034] running on GCE node
W20260807 02:29:37.800877 5647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:37.801724 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:37.802947 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:37.804178 5639 hybrid_clock.cc:648] HybridClock initialized: now 1786069777804126 us; error 64 us; skew 500 ppm
Aug 07 02:29:37 dist-test-slave-g170 krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069777, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:37.812711 5639 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:37.813841 5639 webserver.cc:460] Webserver started at http://127.17.53.62:37559/ using document root <none> and password file <none>
I20260807 02:29:37.814509 5639 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:37.814613 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:37.814909 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:37.817427 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d68cae91ea3b43888320ecf5454f4254"
format_stamp: "Formatted at 2026-08-07 02:29:37 on dist-test-slave-g170"
server_key: "c01018608687f3d43b9fbdc6e3f46244"
server_key_iv: "dc745b080d674fddab835ee698b57053"
server_key_version: "encryptionkey@0"
I20260807 02:29:37.818087 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d68cae91ea3b43888320ecf5454f4254"
format_stamp: "Formatted at 2026-08-07 02:29:37 on dist-test-slave-g170"
server_key: "c01018608687f3d43b9fbdc6e3f46244"
server_key_iv: "dc745b080d674fddab835ee698b57053"
server_key_version: "encryptionkey@0"
I20260807 02:29:37.822358 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:29:37.825045 5666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:37.826336 5639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:29:37.826489 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d68cae91ea3b43888320ecf5454f4254"
format_stamp: "Formatted at 2026-08-07 02:29:37 on dist-test-slave-g170"
server_key: "c01018608687f3d43b9fbdc6e3f46244"
server_key_iv: "dc745b080d674fddab835ee698b57053"
server_key_version: "encryptionkey@0"
I20260807 02:29:37.826606 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:37.850924 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:37.854673 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:37.854908 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:37.862402 5639 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35437
I20260807 02:29:37.862498 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35437 every 8 connection(s)
I20260807 02:29:37.863482 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:37.866427 5746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:37.868570 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5639
I20260807 02:29:37.868676 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:29:37.869040 17620 external_mini_cluster.cc:1428] Setting key ea3a324aacadd9fe11b597ecc9de486e
I20260807 02:29:37.872879 5746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254: Bootstrap starting.
I20260807 02:29:37.875612 5746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:37.876435 5746 log.cc:824] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:29:37 dist-test-slave-g170 krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:37.878605 5746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254: No bootstrap required, opened a new log
I20260807 02:29:37.882701 5746 raft_consensus.cc:348] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } }
I20260807 02:29:37.883164 5746 raft_consensus.cc:374] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:37.883325 5746 raft_consensus.cc:729] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d68cae91ea3b43888320ecf5454f4254, State: Initialized, Role: FOLLOWER
I20260807 02:29:37.884008 5746 consensus_queue.cc:260] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } }
I20260807 02:29:37.884179 5746 raft_consensus.cc:388] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:37.884262 5746 raft_consensus.cc:482] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:37.884405 5746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:37.885196 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:37.870543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.17.53.62:35437)
0807 02:29:37.871019 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:37.871036 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:37.871074 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:29:37.871814 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:37.871834 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:37.872172 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:37.872464 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:37.873501 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:37.874515 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:37.875737 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:37.876235 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:37.876366 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:37.879075 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:37.879080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:37.879094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:37.879121 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:37.881224 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:37.881711 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:37.881717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:37.881722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:37.881846 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:37.882172 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:37.882180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:37.882181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:37.882656 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:37.883585 (+ 929us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:37.883764 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 02:29:37.884141 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":144,"threads_started":1}
I20260807 02:29:37.885591 5746 raft_consensus.cc:504] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } }
I20260807 02:29:37.885948 5746 leader_election.cc:302] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d68cae91ea3b43888320ecf5454f4254; no voters:
I20260807 02:29:37.886363 5746 leader_election.cc:288] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:29:37.886442 5756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:37.886660 5756 raft_consensus.cc:686] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [term 1 LEADER]: Becoming Leader. State: Replica: d68cae91ea3b43888320ecf5454f4254, State: Running, Role: LEADER
I20260807 02:29:37.887063 5756 consensus_queue.cc:237] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } }
I20260807 02:29:37.887753 5746 sys_catalog.cc:564] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:37.888743 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:29:37.888975 5758 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d68cae91ea3b43888320ecf5454f4254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } } }
I20260807 02:29:37.889101 5758 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:37.889400 5757 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d68cae91ea3b43888320ecf5454f4254. Latest consensus state: current_term: 1 leader_uuid: "d68cae91ea3b43888320ecf5454f4254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68cae91ea3b43888320ecf5454f4254" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35437 } } }
I20260807 02:29:37.889480 5757 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:37.889802 5767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:37.892931 5767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:37.898872 5767 catalog_manager.cc:1347] Generated new cluster ID: fd346f68729f4f64bf73a12ad411b41b
I20260807 02:29:37.898958 5767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:37.918924 5767 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:29:37.919925 5767 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:37.930114 5767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d68cae91ea3b43888320ecf5454f4254: Generated new TSK 0
I20260807 02:29:37.930877 5767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:29:37.944967 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5639
2026-08-07T02:29:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1759 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:29:38 dist-test-slave-g170 krb5kdc[5787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:29:38 dist-test-slave-g170 krb5kdc[5787](info): set up 2 sockets
Aug 07 02:29:38 dist-test-slave-g170 krb5kdc[5787](info): commencing operation
krb5kdc: starting...
W20260807 02:29:39.236018 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:29:39 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:29:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:29:39.376181 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:39.490267 5807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:39.490525 5807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:39.490564 5807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:39.494567 5807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:39.494626 5807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:39.494652 5807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:39.494668 5807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:39.494683 5807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:39.499866 5807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:39.500898 5807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:39.501894 5807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:29:39.510660 5807 server_base.cc:1034] running on GCE node
W20260807 02:29:39.510591 5814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:39.510609 5813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:39.510625 5820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:39.511478 5807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:39.512578 5807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:39.513767 5807 hybrid_clock.cc:648] HybridClock initialized: now 1786069779513750 us; error 36 us; skew 500 ppm
Aug 07 02:29:39 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:39.516640 5807 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:39.517618 5807 webserver.cc:460] Webserver started at http://127.17.53.62:34781/ using document root <none> and password file <none>
I20260807 02:29:39.518177 5807 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:39.518285 5807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:39.518530 5807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:39.520541 5807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d507e2aa11324657992ebf3ed2dc72db"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "3023504dd4b5dab95024ddaf424924df"
server_key_iv: "b16244b40779a46a1ef963b36b3ddcfa"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.521153 5807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d507e2aa11324657992ebf3ed2dc72db"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "3023504dd4b5dab95024ddaf424924df"
server_key_iv: "b16244b40779a46a1ef963b36b3ddcfa"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.524930 5807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:29:39.527447 5830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:39.528826 5807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:39.529011 5807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d507e2aa11324657992ebf3ed2dc72db"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "3023504dd4b5dab95024ddaf424924df"
server_key_iv: "b16244b40779a46a1ef963b36b3ddcfa"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.529151 5807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:39.546804 5807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:39.549965 5807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:39.550215 5807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:39.558786 5807 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:44331
I20260807 02:29:39.559165 5907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:44331 every 8 connection(s)
I20260807 02:29:39.560017 5807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:39.562947 5909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:39.563279 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5807
I20260807 02:29:39.563426 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:29:39.563828 17620 external_mini_cluster.cc:1428] Setting key 1a097a67fe9ff0937a0ef78568630ef5
I20260807 02:29:39.570299 5909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap starting.
I20260807 02:29:39.572983 5909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:39.573854 5909 log.cc:824] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:29:39 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:39.576121 5909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: No bootstrap required, opened a new log
I20260807 02:29:39.579582 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:39.565339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.17.53.62:44331)
0807 02:29:39.565764 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:39.565784 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:39.565827 (+ 43us) server_negotiation.cc:407] Connection header received
0807 02:29:39.566577 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:39.566601 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:39.567014 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:39.567382 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:39.568308 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.569806 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.570904 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.571612 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.571772 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:39.574844 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:39.574850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:39.574863 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:39.574890 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:39.576740 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:39.577248 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:39.577258 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:39.577263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:39.577374 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:39.577679 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:39.577682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:39.577684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:39.578067 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:39.578276 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:39.578588 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 02:29:39.578879 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260807 02:29:39.580315 5909 raft_consensus.cc:348] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:39.580590 5909 raft_consensus.cc:374] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:39.580682 5909 raft_consensus.cc:729] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Initialized, Role: FOLLOWER
I20260807 02:29:39.581197 5909 consensus_queue.cc:260] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:39.581398 5909 raft_consensus.cc:388] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:39.581497 5909 raft_consensus.cc:482] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:39.581660 5909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:39.582693 5909 raft_consensus.cc:504] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:39.583078 5909 leader_election.cc:302] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d507e2aa11324657992ebf3ed2dc72db; no voters:
I20260807 02:29:39.583447 5909 leader_election.cc:288] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:29:39.583540 5918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:39.583871 5918 raft_consensus.cc:686] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 LEADER]: Becoming Leader. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Running, Role: LEADER
I20260807 02:29:39.584265 5918 consensus_queue.cc:237] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:39.584967 5909 sys_catalog.cc:564] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:39.586352 5923 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: New leader d507e2aa11324657992ebf3ed2dc72db. Latest consensus state: current_term: 1 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:39.586351 5922 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:39.586515 5923 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:39.586517 5922 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:39.587327 5931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:39.590549 5931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:39.597052 5931 catalog_manager.cc:1347] Generated new cluster ID: e1efa402e60f4117888c725225530dc3
I20260807 02:29:39.597158 5931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:39.612363 5931 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:29:39.613474 5931 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:39.620949 5931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Generated new TSK 0
I20260807 02:29:39.621956 5931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:29:39.662257 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:39.782773 5951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:39.783056 5951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:39.783145 5951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:39.787420 5951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:39.787505 5951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:39.787595 5951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:39.792896 5951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:39.794039 5951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:39.795121 5951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:39.804912 5957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:39.804963 5962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:39.804924 5959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:39.805367 5951 server_base.cc:1034] running on GCE node
I20260807 02:29:39.805755 5951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:39.806388 5951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:39.807592 5951 hybrid_clock.cc:648] HybridClock initialized: now 1786069779807566 us; error 36 us; skew 500 ppm
Aug 07 02:29:39 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:39.810492 5951 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:39.811399 5951 webserver.cc:460] Webserver started at http://127.17.53.1:33991/ using document root <none> and password file <none>
I20260807 02:29:39.811982 5951 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:39.812074 5951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:39.812318 5951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:39.814510 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "2390301cfc674befadd94e96df100198"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "22847081187d5616b8f5495021d1bd30"
server_key_iv: "f1e4f5d79bd39526baef6f25631c22c5"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.815184 5951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "2390301cfc674befadd94e96df100198"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "22847081187d5616b8f5495021d1bd30"
server_key_iv: "f1e4f5d79bd39526baef6f25631c22c5"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.821082 5951 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 02:29:39.824492 5973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:39.825969 5951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:39.826115 5951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "2390301cfc674befadd94e96df100198"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "22847081187d5616b8f5495021d1bd30"
server_key_iv: "f1e4f5d79bd39526baef6f25631c22c5"
server_key_version: "encryptionkey@0"
I20260807 02:29:39.826239 5951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:39.862609 5951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:39.866222 5951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:39.866464 5951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:39.867141 5951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:39.868022 5951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:39.868072 5951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:39.868113 5951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:39.868168 5951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:39.878304 5951 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:44419
I20260807 02:29:39.878324 6147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:44419 every 8 connection(s)
I20260807 02:29:39.879364 5951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:39.881078 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 5951
I20260807 02:29:39.881245 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:29:39.881639 17620 external_mini_cluster.cc:1428] Setting key 08ae5aab32577c3c92df637a0bfb971a
Aug 07 02:29:39 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069779, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:29:39.896915 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:39.881650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:41197 (local address 127.17.53.62:44331)
0807 02:29:39.881845 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:39.881850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:39.882931 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 02:29:39.885245 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:39.885248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:39.885294 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:39.885367 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:39.886903 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.887774 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.889208 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.889609 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.889796 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:39.892542 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:39.892550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:39.892555 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:39.892592 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:39.894225 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:39.895010 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:39.895014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:39.895015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:39.895117 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:39.895524 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:39.895526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:39.895528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:39.895807 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:39.895899 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:39.896504 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 02:29:39.896661 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 02:29:39.897995 6153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:39.882171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:41197)
0807 02:29:39.882744 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:39.882805 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:39.884986 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:39.885601 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:39.885614 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:39.886158 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:39.886697 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:39.886740 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.887931 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.887934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:39.889049 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:39.889061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:39.889791 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:39.889794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:39.889974 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:39.890711 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:39.890729 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:39.892349 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:39.894377 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:39.894383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:39.894397 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:39.894879 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:39.895245 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:39.895247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:39.895249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:39.895422 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:39.895919 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:39.895925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:39.896173 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 02:29:39.896402 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 02:29:39.896751 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":167,"threads_started":1}
I20260807 02:29:39.899931 6149 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:39.900241 6149 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:39.900851 6149 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:39.902777 5859 ts_manager.cc:194] Registered new tserver with Master: 2390301cfc674befadd94e96df100198 (127.17.53.1:44419)
I20260807 02:29:39.904559 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:41197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:29:39.916455 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:40.035197 6156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:40.035454 6156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:40.035593 6156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:40.040179 6156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:40.040241 6156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:40.040347 6156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:40.046217 6156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:40.047536 6156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:40.048866 6156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:40.059677 6163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.059677 6168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.059836 6156 server_base.cc:1034] running on GCE node
W20260807 02:29:40.059698 6164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.060549 6156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:40.061259 6156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:40.062469 6156 hybrid_clock.cc:648] HybridClock initialized: now 1786069780062457 us; error 30 us; skew 500 ppm
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:40.065412 6156 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:40.066401 6156 webserver.cc:460] Webserver started at http://127.17.53.2:32893/ using document root <none> and password file <none>
I20260807 02:29:40.067013 6156 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:40.067116 6156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:40.067384 6156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:40.069682 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "f74c5f83ddc84a28a22959390667ab8a"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "763cc559e54f9e5b4510253e1a599f65"
server_key_iv: "83bb3884b35ecc4c18acbb83dd69a931"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.070307 6156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "f74c5f83ddc84a28a22959390667ab8a"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "763cc559e54f9e5b4510253e1a599f65"
server_key_iv: "83bb3884b35ecc4c18acbb83dd69a931"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.074406 6156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:29:40.077025 6178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.078307 6156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:29:40.078461 6156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "f74c5f83ddc84a28a22959390667ab8a"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "763cc559e54f9e5b4510253e1a599f65"
server_key_iv: "83bb3884b35ecc4c18acbb83dd69a931"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.078581 6156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:40.114948 6156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:40.118144 6156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:40.118453 6156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:40.119156 6156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:40.120090 6156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:40.120191 6156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.120297 6156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:40.120357 6156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.130584 6156 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40721
I20260807 02:29:40.130613 6345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40721 every 8 connection(s)
I20260807 02:29:40.131690 6156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:40.134393 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 6156
I20260807 02:29:40.134521 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:29:40.134820 17620 external_mini_cluster.cc:1428] Setting key 5c16ef73cf65b4716f3a0f143073b54f
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:29:40.147732 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.133679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:56613 (local address 127.17.53.62:44331)
0807 02:29:40.133874 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.133879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.134906 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 02:29:40.136682 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.136689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.136741 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.136866 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:40.138012 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.138806 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.140504 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.140944 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.141078 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.143570 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:40.143576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:40.143581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:40.143611 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:40.145372 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.145964 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.145968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.145970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.146061 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.146534 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.146538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.146542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.146772 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:40.146850 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.147487 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.147607 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 02:29:40.148706 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.134230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:56613)
0807 02:29:40.134746 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.134803 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.136468 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.137018 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.137025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.137381 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:40.137866 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.137880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.138942 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.138948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.140372 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.140383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.141092 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.141094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.141254 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.141928 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:40.141945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:40.143395 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:40.145522 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:40.145527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:40.145540 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:40.145847 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:40.146182 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:40.146184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:40.146186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:40.146333 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:40.146900 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:40.146906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:40.147186 (+ 280us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.147411 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.147708 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":147,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:29:40.150141 6346 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:40.150425 6346 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:40.151049 6346 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:40.152280 5859 ts_manager.cc:194] Registered new tserver with Master: f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721)
I20260807 02:29:40.153108 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:56613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:29:40.167572 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:40.288929 6351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:40.289183 6351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:40.289227 6351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:40.293164 6351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:40.293216 6351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:40.293272 6351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:29:40.298427 6351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:40.299430 6351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:40.300423 6351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:40.310323 6364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.310303 6362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.310297 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.310408 6351 server_base.cc:1034] running on GCE node
I20260807 02:29:40.311040 6351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:40.311635 6351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:40.312856 6351 hybrid_clock.cc:648] HybridClock initialized: now 1786069780312819 us; error 43 us; skew 500 ppm
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:40.315847 6351 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:29:40.316753 6351 webserver.cc:460] Webserver started at http://127.17.53.3:35135/ using document root <none> and password file <none>
I20260807 02:29:40.317364 6351 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:40.317431 6351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:40.317600 6351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:40.319674 6351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "740dcc6befed431eb04ff6da18e7ede6"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "e47d6987ac200617b86d8f5230fa02a4"
server_key_iv: "16dbaefce2300999fe36f826616fa20a"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.320214 6351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "740dcc6befed431eb04ff6da18e7ede6"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "e47d6987ac200617b86d8f5230fa02a4"
server_key_iv: "16dbaefce2300999fe36f826616fa20a"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.324095 6351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 02:29:40.326603 6372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.328209 6351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:29:40.328356 6351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "740dcc6befed431eb04ff6da18e7ede6"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "e47d6987ac200617b86d8f5230fa02a4"
server_key_iv: "16dbaefce2300999fe36f826616fa20a"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.328465 6351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:40.344094 6351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:40.347885 6351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:40.348134 6351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:40.348851 6351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:40.349900 6351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:40.349972 6351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.350034 6351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:40.350087 6351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.359968 6351 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:42681
I20260807 02:29:40.359994 6549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:42681 every 8 connection(s)
I20260807 02:29:40.360914 6351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:29:40.365010 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 6351
I20260807 02:29:40.365134 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:29:40.365446 17620 external_mini_cluster.cc:1428] Setting key ce5743ad860a2c3d9247a5781ad0288e
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:40.375849 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.362838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:44401 (local address 127.17.53.62:44331)
0807 02:29:40.362994 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.362998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.363840 (+ 842us) server_negotiation.cc:407] Connection header received
0807 02:29:40.365621 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.365624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.365676 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.365754 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:40.367360 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.368005 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.369266 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.369495 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.369607 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.372301 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:40.372305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:40.372307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:40.372333 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:40.373745 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.374278 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.374281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.374282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.374355 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.374731 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.374735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.374737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.374974 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:40.375077 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.375604 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.375716 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:29:40.376857 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.363166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:44401)
0807 02:29:40.363654 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.363699 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.365337 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.365965 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.365972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.366503 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:40.367206 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.367222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.368135 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.368139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.369118 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.369132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.369651 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.369655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.369830 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.370465 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:40.370483 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:40.372110 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:40.373874 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:40.373879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:40.373892 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:40.374179 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:40.374441 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:40.374443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:40.374445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:40.374598 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:40.375102 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:40.375108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:40.375357 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.375539 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.375861 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260807 02:29:40.378068 6550 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:40.378371 6550 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:40.378948 6550 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:40.380095 5859 ts_manager.cc:194] Registered new tserver with Master: 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
I20260807 02:29:40.380798 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:44401
I20260807 02:29:40.391608 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:29:40.401543 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.393764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.17.53.62:44331)
0807 02:29:40.393938 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.393943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.394112 (+ 169us) server_negotiation.cc:407] Connection header received
0807 02:29:40.394267 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.394270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.394311 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.394397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:40.395137 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.395645 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.396519 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.396736 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.396871 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.398253 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:40.398268 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:40.398270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:40.398293 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:40.399756 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.400182 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.400185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.400187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.400278 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:40.400561 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:40.400564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:40.400566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:40.400852 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:40.400991 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.401241 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.401380 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 02:29:40.404286 5859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:29:40.407256 5859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:29:40.422374 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.417309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42681 (local address 127.0.0.1:36296)
0807 02:29:40.417955 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.417967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.418044 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.418648 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.418653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.418668 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:40.418739 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.418744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.420043 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.420046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.420951 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.420962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.421876 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.421879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.421954 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.421985 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.422111 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.422229 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":82,"threads_started":1}
I20260807 02:29:40.422410 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.417576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.17.53.3:42681)
0807 02:29:40.418122 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.418134 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.418157 (+ 23us) server_negotiation.cc:407] Connection header received
0807 02:29:40.418232 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.418238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.418446 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.418596 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:40.418843 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.419907 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.421075 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.421760 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.421871 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.422010 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.422154 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.422247 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":205,"threads_started":1}
I20260807 02:29:40.423309 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.416411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40721 (local address 127.0.0.1:47294)
0807 02:29:40.416819 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.416873 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.417022 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.418044 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.418050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.418077 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:40.418232 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.418240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.419806 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.419812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.421395 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.421408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.422883 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.422887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.422977 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.423001 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.423073 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.423143 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":95,"thread_start_us":92,"threads_started":1}
I20260807 02:29:40.423771 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.416576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47294 (local address 127.17.53.2:40721)
0807 02:29:40.417314 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.417328 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.417354 (+ 26us) server_negotiation.cc:407] Connection header received
0807 02:29:40.417441 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.417446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.417656 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.417835 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:40.418386 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.419637 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.421534 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.422598 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.422690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.422820 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.423493 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.423607 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":140,"threads_started":1}
I20260807 02:29:40.424086 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.417185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44419 (local address 127.0.0.1:43680)
0807 02:29:40.417655 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.417670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.417768 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.418390 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.418392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.418408 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:40.418508 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.418517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.419862 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.419864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.421395 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.421408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.423623 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.423627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.423739 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.423761 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.423859 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.423947 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":152,"thread_start_us":109,"threads_started":1}
I20260807 02:29:40.424638 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.417518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.17.53.1:44419)
0807 02:29:40.417932 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.417938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.417962 (+ 24us) server_negotiation.cc:407] Connection header received
0807 02:29:40.418044 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.418050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.418237 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.418408 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:40.418629 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.419739 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.422089 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.423248 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.423401 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.423544 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.424075 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.424499 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":95,"threads_started":1}
I20260807 02:29:40.425771 6446 tablet_service.cc:1467] Processing CreateTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 (DEFAULT_TABLE table=test-table [id=1676d972ea4c4ab19f5f4551cf357c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:40.426417 6248 tablet_service.cc:1467] Processing CreateTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 (DEFAULT_TABLE table=test-table [id=1676d972ea4c4ab19f5f4551cf357c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:40.426992 6446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439b1a1d96d441c68cbdc438845ebf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:40.427433 6248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439b1a1d96d441c68cbdc438845ebf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:40.427446 6039 tablet_service.cc:1467] Processing CreateTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 (DEFAULT_TABLE table=test-table [id=1676d972ea4c4ab19f5f4551cf357c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:40.428383 6039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 439b1a1d96d441c68cbdc438845ebf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:40.433944 6573 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Bootstrap starting.
I20260807 02:29:40.435511 6576 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Bootstrap starting.
I20260807 02:29:40.436107 6574 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Bootstrap starting.
I20260807 02:29:40.436843 6573 tablet_bootstrap.cc:654] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:40.437750 6573 log.cc:824] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:40.438148 6576 tablet_bootstrap.cc:654] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:40.438162 6574 tablet_bootstrap.cc:654] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:40.438962 6576 log.cc:824] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:40.439370 6574 log.cc:824] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:40.440058 6573 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: No bootstrap required, opened a new log
I20260807 02:29:40.440263 6573 ts_tablet_manager.cc:1397] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 02:29:40.440975 6576 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: No bootstrap required, opened a new log
I20260807 02:29:40.441160 6576 ts_tablet_manager.cc:1397] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 02:29:40.441176 6574 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: No bootstrap required, opened a new log
I20260807 02:29:40.441367 6574 ts_tablet_manager.cc:1397] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260807 02:29:40.444311 6574 raft_consensus.cc:348] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.444298 6573 raft_consensus.cc:348] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.444561 6574 raft_consensus.cc:374] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:40.444561 6573 raft_consensus.cc:374] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:40.444615 6574 raft_consensus.cc:729] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740dcc6befed431eb04ff6da18e7ede6, State: Initialized, Role: FOLLOWER
I20260807 02:29:40.444618 6573 raft_consensus.cc:729] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74c5f83ddc84a28a22959390667ab8a, State: Initialized, Role: FOLLOWER
I20260807 02:29:40.445060 6574 consensus_queue.cc:260] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.445143 6573 consensus_queue.cc:260] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.445492 6576 raft_consensus.cc:348] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.445717 6550 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:40.445739 6576 raft_consensus.cc:374] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:40.445788 6576 raft_consensus.cc:729] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2390301cfc674befadd94e96df100198, State: Initialized, Role: FOLLOWER
I20260807 02:29:40.446096 6574 ts_tablet_manager.cc:1428] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 02:29:40.446169 6576 consensus_queue.cc:260] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.446770 6149 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:40.447192 6576 ts_tablet_manager.cc:1428] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:29:40.447222 6573 ts_tablet_manager.cc:1428] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 02:29:40.447544 6346 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:40.507601 6584 raft_consensus.cc:482] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:40.507845 6584 raft_consensus.cc:504] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.509671 6584 leader_election.cc:288] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
I20260807 02:29:40.513772 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.509706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:51925 (local address 127.17.53.3:42681)
0807 02:29:40.509865 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.509870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.510390 (+ 520us) server_negotiation.cc:407] Connection header received
0807 02:29:40.510581 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.510587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.510642 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.510723 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:40.511047 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.511576 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.512737 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.513159 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.513247 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.513342 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.513544 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.513624 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 02:29:40.513789 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.509628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40721 (local address 127.17.53.1:37641)
0807 02:29:40.510119 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.510134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.510237 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.510678 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.510680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.510696 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:40.510815 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.510819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.511625 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.511631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.512509 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.512519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.513357 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.513362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.513445 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.513463 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.513562 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.513626 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":183,"threads_started":1}
I20260807 02:29:40.513803 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.510025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:37641 (local address 127.17.53.2:40721)
0807 02:29:40.510171 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.510176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.510194 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:29:40.510396 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.510400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.510460 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.510512 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:40.510937 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.511493 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.512753 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.513191 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.513326 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.513394 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.513601 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.513688 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 02:29:40.513806 6586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.509638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42681 (local address 127.17.53.1:51925)
0807 02:29:40.510231 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.510246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:40.510395 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:40.510754 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:40.510757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:40.510775 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:40.510925 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.510932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.511705 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.511707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.512511 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:40.512520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.513284 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.513291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:40.513391 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.513425 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 02:29:40.513500 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 02:29:40.513589 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":172,"thread_start_us":120,"threads_started":1}
I20260807 02:29:40.514658 6277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439b1a1d96d441c68cbdc438845ebf19" candidate_uuid: "2390301cfc674befadd94e96df100198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74c5f83ddc84a28a22959390667ab8a" is_pre_election: true
I20260807 02:29:40.514657 6476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439b1a1d96d441c68cbdc438845ebf19" candidate_uuid: "2390301cfc674befadd94e96df100198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "740dcc6befed431eb04ff6da18e7ede6" is_pre_election: true
I20260807 02:29:40.514978 6476 raft_consensus.cc:2454] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2390301cfc674befadd94e96df100198 in term 0.
I20260807 02:29:40.514992 6277 raft_consensus.cc:2454] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2390301cfc674befadd94e96df100198 in term 0.
I20260807 02:29:40.515439 5976 leader_election.cc:302] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2390301cfc674befadd94e96df100198, 740dcc6befed431eb04ff6da18e7ede6; no voters:
I20260807 02:29:40.515729 6584 raft_consensus.cc:2781] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:29:40.515821 6584 raft_consensus.cc:482] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:29:40.515866 6584 raft_consensus.cc:3037] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:40.516997 6584 raft_consensus.cc:504] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.517349 6584 leader_election.cc:288] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 1 election: Requested vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
I20260807 02:29:40.517697 6476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439b1a1d96d441c68cbdc438845ebf19" candidate_uuid: "2390301cfc674befadd94e96df100198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "740dcc6befed431eb04ff6da18e7ede6"
I20260807 02:29:40.517804 6476 raft_consensus.cc:3037] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:40.517819 6277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "439b1a1d96d441c68cbdc438845ebf19" candidate_uuid: "2390301cfc674befadd94e96df100198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74c5f83ddc84a28a22959390667ab8a"
I20260807 02:29:40.517966 6277 raft_consensus.cc:3037] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:40.518952 6476 raft_consensus.cc:2454] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2390301cfc674befadd94e96df100198 in term 1.
I20260807 02:29:40.519222 6277 raft_consensus.cc:2454] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2390301cfc674befadd94e96df100198 in term 1.
I20260807 02:29:40.519251 5976 leader_election.cc:302] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2390301cfc674befadd94e96df100198, 740dcc6befed431eb04ff6da18e7ede6; no voters:
I20260807 02:29:40.519407 6584 raft_consensus.cc:2781] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:40.519641 6584 raft_consensus.cc:686] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 LEADER]: Becoming Leader. State: Replica: 2390301cfc674befadd94e96df100198, State: Running, Role: LEADER
I20260807 02:29:40.520004 6584 consensus_queue.cc:237] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:40.523301 5859 catalog_manager.cc:5495] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2390301cfc674befadd94e96df100198 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "2390301cfc674befadd94e96df100198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } health_report { overall_health: UNKNOWN } } }
I20260807 02:29:40.541983 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.538635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.17.53.1:44419)
0807 02:29:40.538754 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:40.538758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:40.538774 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:29:40.538930 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:40.538932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:40.538985 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:40.539073 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:29:40.539321 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.539806 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.540604 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:40.540860 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:40.540937 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:40.541191 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:29:40.541591 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:29:40.541682 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:40.541829 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 02:29:40.541881 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:29:40.548038 6476 raft_consensus.cc:1261] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Refusing update from remote peer 2390301cfc674befadd94e96df100198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:40.548038 6277 raft_consensus.cc:1261] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Refusing update from remote peer 2390301cfc674befadd94e96df100198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:40.548900 6584 consensus_queue.cc:1035] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:40.549110 6588 consensus_queue.cc:1035] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:40.566325 5857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:40.566502 5857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:40.569206 5857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Sending DeleteTablet for 3 replicas of tablet 439b1a1d96d441c68cbdc438845ebf19
I20260807 02:29:40.570226 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5951
I20260807 02:29:40.570348 6446 tablet_service.cc:1514] Processing DeleteTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:40 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:36296
I20260807 02:29:40.570847 6248 tablet_service.cc:1514] Processing DeleteTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:40 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47294
I20260807 02:29:40.572805 6610 tablet_replica.cc:330] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: stopping tablet replica
I20260807 02:29:40.573101 6610 raft_consensus.cc:2229] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:40.573369 6610 raft_consensus.cc:2258] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:40.577697 6609 tablet_replica.cc:330] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: stopping tablet replica
I20260807 02:29:40.578027 6609 raft_consensus.cc:2229] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:40.578312 6609 raft_consensus.cc:2258] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:40.579377 6610 ts_tablet_manager.cc:1905] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:40.581691 6610 ts_tablet_manager.cc:1918] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:40.581821 6610 log.cc:1196] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/439b1a1d96d441c68cbdc438845ebf19
I20260807 02:29:40.582173 6610 ts_tablet_manager.cc:1939] T 439b1a1d96d441c68cbdc438845ebf19 P f74c5f83ddc84a28a22959390667ab8a: Deleting consensus metadata
I20260807 02:29:40.583473 5833 catalog_manager.cc:4841] TS f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): tablet 439b1a1d96d441c68cbdc438845ebf19 (table test-table [id=1676d972ea4c4ab19f5f4551cf357c89]) successfully deleted
I20260807 02:29:40.584430 6609 ts_tablet_manager.cc:1905] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:40.586915 6609 ts_tablet_manager.cc:1918] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:40.586997 6609 log.cc:1196] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/439b1a1d96d441c68cbdc438845ebf19
I20260807 02:29:40.587314 6609 ts_tablet_manager.cc:1939] T 439b1a1d96d441c68cbdc438845ebf19 P 740dcc6befed431eb04ff6da18e7ede6: Deleting consensus metadata
W20260807 02:29:40.588361 5833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:44419 (error 108)
I20260807 02:29:40.588579 5832 catalog_manager.cc:4841] TS 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): tablet 439b1a1d96d441c68cbdc438845ebf19 (table test-table [id=1676d972ea4c4ab19f5f4551cf357c89]) successfully deleted
I20260807 02:29:40.588691 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 6156
I20260807 02:29:40.590936 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:40.590642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44419 (local address 127.0.0.1:43702)
0807 02:29:40.590783 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:40.590862 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 02:29:40.591133 5833 catalog_manager.cc:4584] TS 2390301cfc674befadd94e96df100198 (127.17.53.1:44419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 439b1a1d96d441c68cbdc438845ebf19: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44419: connect: Connection refused (error 111)
I20260807 02:29:40.599136 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 6351
I20260807 02:29:40.615561 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 5807
I20260807 02:29:40.626713 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--webserver_interface=127.17.53.62
--webserver_port=34781
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:40.745332 6612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:40.745651 6612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:40.745739 6612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:40.750258 6612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:40.750346 6612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:40.750367 6612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:40.750393 6612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:40.750458 6612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:40.756302 6612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=34781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:40.757670 6612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:40.758852 6612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:40.773861 6624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.773947 6612 server_base.cc:1034] running on GCE node
W20260807 02:29:40.773900 6622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.773861 6621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.774720 6612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:40.775985 6612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:40.777176 6612 hybrid_clock.cc:648] HybridClock initialized: now 1786069780777162 us; error 27 us; skew 500 ppm
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:40.780539 6612 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:40.781931 6612 webserver.cc:460] Webserver started at http://127.17.53.62:34781/ using document root <none> and password file <none>
I20260807 02:29:40.782753 6612 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:40.782919 6612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:40.792325 6612 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.004s
I20260807 02:29:40.795318 6634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:40.796739 6612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:29:40.796983 6612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d507e2aa11324657992ebf3ed2dc72db"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "3023504dd4b5dab95024ddaf424924df"
server_key_iv: "b16244b40779a46a1ef963b36b3ddcfa"
server_key_version: "encryptionkey@0"
I20260807 02:29:40.797662 6612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:40.822222 6612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:40.825901 6612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:40.826136 6612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:40.834596 6612 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:44331
I20260807 02:29:40.834973 6716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:44331 every 8 connection(s)
I20260807 02:29:40.835853 6612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:40.840745 6720 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:29:40.842634 6720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap starting.
I20260807 02:29:40.844856 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 6612
I20260807 02:29:40.845350 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:44419
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=33991
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:40.861670 6720 log.cc:824] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:40.871012 6720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:29:40.871593 6720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap complete.
I20260807 02:29:40.875954 6720 raft_consensus.cc:348] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:40.876405 6720 raft_consensus.cc:729] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Initialized, Role: FOLLOWER
I20260807 02:29:40.876995 6720 consensus_queue.cc:260] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:40.877153 6720 raft_consensus.cc:388] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:40.877250 6720 raft_consensus.cc:482] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:40.877379 6720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:29:40.879334 6720 raft_consensus.cc:504] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:40.879688 6720 leader_election.cc:302] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d507e2aa11324657992ebf3ed2dc72db; no voters:
I20260807 02:29:40.880075 6720 leader_election.cc:288] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 02:29:40.880170 6732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Leader election won for term 2
I20260807 02:29:40.880383 6732 raft_consensus.cc:686] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 LEADER]: Becoming Leader. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Running, Role: LEADER
I20260807 02:29:40.880693 6732 consensus_queue.cc:237] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:40.881410 6720 sys_catalog.cc:564] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:40.882103 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:40.882241 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:40.882905 6735 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: New leader d507e2aa11324657992ebf3ed2dc72db. Latest consensus state: current_term: 2 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:40.883081 6735 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:40.883348 6740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:40.888329 6740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:40.888969 6740 catalog_manager.cc:1259] Loaded cluster ID: e1efa402e60f4117888c725225530dc3
I20260807 02:29:40.889032 6740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:40.890491 6740 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 02:29:40.895138 6740 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 02:29:40.900605 6740 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:40.901427 6740 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Loaded TSK: 0
I20260807 02:29:40.902648 6740 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:29:40.970988 6727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:40.971226 6727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:40.971276 6727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:40.975435 6727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:40.975492 6727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:40.975575 6727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:40.980865 6727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:44419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=33991
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:40.981922 6727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:40.983071 6727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:40.993934 6769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.993927 6772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:40.993927 6767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:40.994405 6727 server_base.cc:1034] running on GCE node
I20260807 02:29:40.994804 6727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:40.995472 6727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:40.996655 6727 hybrid_clock.cc:648] HybridClock initialized: now 1786069780996624 us; error 48 us; skew 500 ppm
Aug 07 02:29:40 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:40.999507 6727 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:41.000458 6727 webserver.cc:460] Webserver started at http://127.17.53.1:33991/ using document root <none> and password file <none>
I20260807 02:29:41.001118 6727 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:41.001194 6727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:41.005242 6727 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:29:41.007467 6780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.008893 6727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:41.009054 6727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "2390301cfc674befadd94e96df100198"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "22847081187d5616b8f5495021d1bd30"
server_key_iv: "f1e4f5d79bd39526baef6f25631c22c5"
server_key_version: "encryptionkey@0"
I20260807 02:29:41.009621 6727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:41.041581 6727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:41.044740 6727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:41.045019 6727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:41.045644 6727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:41.046772 6793 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 02:29:41.051154 6727 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 02:29:41.051213 6727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 02:29:41.051286 6727 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 02:29:41.053012 6727 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 02:29:41.053057 6727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 02:29:41.053361 6793 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Bootstrap starting.
I20260807 02:29:41.064121 6727 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:44419
I20260807 02:29:41.064180 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:44419 every 8 connection(s)
I20260807 02:29:41.065137 6727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:41.073568 6793 log.cc:824] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:41.073611 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 6727
I20260807 02:29:41.074119 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:40721
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=32893
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:41 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069780, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:41.083149 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.067343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:35483 (local address 127.17.53.62:44331)
0807 02:29:41.067882 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.067903 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.068053 (+ 150us) server_negotiation.cc:407] Connection header received
0807 02:29:41.069454 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.069465 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.069747 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.070046 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:41.071595 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.072548 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.074199 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.074859 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.075030 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.077920 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:41.077927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:41.077943 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:41.077972 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:41.079807 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.080364 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.080369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.080374 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.080491 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.080884 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.080887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.080888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.081219 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:41.081366 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.082001 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.082273 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":126,"threads_started":1}
I20260807 02:29:41.083163 6958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.067229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:35483)
0807 02:29:41.067845 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.067917 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.069050 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.070182 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.070191 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.070668 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:41.071413 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.071441 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.072718 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.072721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.074062 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.074072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.075244 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.075248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.075424 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.076139 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:41.076165 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:41.077692 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:41.079942 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.079947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.079960 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.080221 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.080593 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.080596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.080598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.080743 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.081340 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:41.081346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:41.081679 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.081863 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.082203 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":143,"threads_started":1}
I20260807 02:29:41.084249 6793 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:29:41.084791 6793 tablet_bootstrap.cc:492] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Bootstrap complete.
I20260807 02:29:41.085815 6955 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:41.086061 6955 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:41.086611 6955 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:41.086618 6793 ts_tablet_manager.cc:1397] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.013s
I20260807 02:29:41.089627 6658 ts_manager.cc:194] Registered new tserver with Master: 2390301cfc674befadd94e96df100198 (127.17.53.1:44419)
I20260807 02:29:41.090171 6793 raft_consensus.cc:348] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:41.090734 6793 raft_consensus.cc:729] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2390301cfc674befadd94e96df100198, State: Initialized, Role: FOLLOWER
I20260807 02:29:41.091236 6793 consensus_queue.cc:260] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } }
I20260807 02:29:41.092267 6793 ts_tablet_manager.cc:1428] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:29:41.093665 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:35483
I20260807 02:29:41.096388 6955 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:41.100589 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.093090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.17.53.1:44419)
0807 02:29:41.095037 (+ 1947us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.095049 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.095070 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:29:41.095139 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.095144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.095527 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.095691 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:41.096017 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.097749 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.099410 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.099949 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.100037 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.100206 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.100413 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.100487 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1854,"thread_start_us":95,"threads_started":1}
I20260807 02:29:41.100680 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.093019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44419 (local address 127.0.0.1:43710)
0807 02:29:41.094581 (+ 1562us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.094614 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.094767 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.095723 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.095730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.095759 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:41.095888 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.095894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.097888 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.097891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.099279 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.099291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.100068 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.100072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.100212 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.100262 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.100378 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.100505 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1395,"thread_start_us":124,"threads_started":1}
I20260807 02:29:41.101745 6859 tablet_service.cc:1514] Processing DeleteTablet for tablet 439b1a1d96d441c68cbdc438845ebf19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:40 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:43710
I20260807 02:29:41.102648 6968 tablet_replica.cc:330] stopping tablet replica
I20260807 02:29:41.102945 6968 raft_consensus.cc:2229] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:41.103185 6968 raft_consensus.cc:2258] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:41.104133 6968 ts_tablet_manager.cc:1905] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:41.107612 6968 ts_tablet_manager.cc:1918] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:41.107693 6968 log.cc:1196] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/439b1a1d96d441c68cbdc438845ebf19
I20260807 02:29:41.107986 6968 ts_tablet_manager.cc:1939] T 439b1a1d96d441c68cbdc438845ebf19 P 2390301cfc674befadd94e96df100198: Deleting consensus metadata
I20260807 02:29:41.109653 6639 catalog_manager.cc:4841] TS 2390301cfc674befadd94e96df100198 (127.17.53.1:44419): tablet 439b1a1d96d441c68cbdc438845ebf19 (table test-table [id=1676d972ea4c4ab19f5f4551cf357c89]) successfully deleted
W20260807 02:29:41.198103 6963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:41.198352 6963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:41.198416 6963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:41.202973 6963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:41.203032 6963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:41.203092 6963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:41.208352 6963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:40721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=32893
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:41.209424 6963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:41.210414 6963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:41.220471 6982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:41.220547 6963 server_base.cc:1034] running on GCE node
W20260807 02:29:41.220497 6974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:41.220487 6977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:41.221180 6963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:41.221817 6963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:41.223007 6963 hybrid_clock.cc:648] HybridClock initialized: now 1786069781222978 us; error 38 us; skew 500 ppm
Aug 07 02:29:41 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:41.225968 6963 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:41.226936 6963 webserver.cc:460] Webserver started at http://127.17.53.2:32893/ using document root <none> and password file <none>
I20260807 02:29:41.227530 6963 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:41.227664 6963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:41.231719 6963 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:29:41.234366 6990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.235813 6963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:29:41.235980 6963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "f74c5f83ddc84a28a22959390667ab8a"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "763cc559e54f9e5b4510253e1a599f65"
server_key_iv: "83bb3884b35ecc4c18acbb83dd69a931"
server_key_version: "encryptionkey@0"
I20260807 02:29:41.236536 6963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:41.253571 6963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:41.257184 6963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:41.257390 6963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:41.257968 6963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:41.258855 6963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:41.258906 6963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.258972 6963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:41.259011 6963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.267767 6963 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40721
I20260807 02:29:41.267818 7163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40721 every 8 connection(s)
I20260807 02:29:41.268908 6963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:41.273013 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 6963
I20260807 02:29:41.273439 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:42681
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=35135
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:29:41 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:41.285032 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.270715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:44729 (local address 127.17.53.62:44331)
0807 02:29:41.270864 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.270868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.271707 (+ 839us) server_negotiation.cc:407] Connection header received
0807 02:29:41.273325 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.273329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.273387 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.273500 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:41.274662 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.275261 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.277099 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.277342 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.277434 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.280524 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:41.280527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:41.280529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:41.280555 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:41.282620 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.283144 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.283150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.283155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.283272 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.283634 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.283640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.283644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.283938 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:41.284077 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.284563 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.284735 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:29:41.285641 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.271096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:44729)
0807 02:29:41.271558 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.271601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.273069 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.273686 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.273694 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.274042 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:41.274510 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.274524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.275440 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.275447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.276966 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.276981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.277516 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.277523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.277770 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.278414 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:41.278446 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:41.280334 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:41.282752 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.282757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.282770 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.283021 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.283388 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.283391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.283392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.283524 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.284064 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:41.284069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:41.284300 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.284454 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.284693 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
I20260807 02:29:41.286958 7164 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:41.287259 7164 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:41.287933 7164 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:41.289198 6658 ts_manager.cc:194] Registered new tserver with Master: f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721)
I20260807 02:29:41.289990 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:44729
W20260807 02:29:41.395299 7168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:41.395538 7168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:41.395579 7168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:41.399523 7168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:41.399570 7168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:41.399626 7168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:29:41.404749 7168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:42681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=35135
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:41.405761 7168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:41.406718 7168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:41.416752 7177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:41.416880 7178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:41.416996 7180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:41.417018 7168 server_base.cc:1034] running on GCE node
I20260807 02:29:41.417549 7168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:41.418203 7168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:41.419404 7168 hybrid_clock.cc:648] HybridClock initialized: now 1786069781419374 us; error 39 us; skew 500 ppm
Aug 07 02:29:41 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:41.422194 7168 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:29:41.423137 7168 webserver.cc:460] Webserver started at http://127.17.53.3:35135/ using document root <none> and password file <none>
I20260807 02:29:41.423677 7168 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:41.423751 7168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:41.427524 7168 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:29:41.429531 7191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.430703 7168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:29:41.430840 7168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "740dcc6befed431eb04ff6da18e7ede6"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "e47d6987ac200617b86d8f5230fa02a4"
server_key_iv: "16dbaefce2300999fe36f826616fa20a"
server_key_version: "encryptionkey@0"
I20260807 02:29:41.431339 7168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:41.442071 7168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:41.445068 7168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:41.445261 7168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:41.445834 7168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:41.446628 7168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:41.446676 7168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.446714 7168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:41.446753 7168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:41.455910 7168 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:42681
I20260807 02:29:41.455936 7351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:42681 every 8 connection(s)
I20260807 02:29:41.457002 7168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:29:41.461870 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 7168
Aug 07 02:29:41 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069781, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:41.471364 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.462606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.17.53.62:44331)
0807 02:29:41.462953 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.462958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.463006 (+ 48us) server_negotiation.cc:407] Connection header received
0807 02:29:41.463136 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.463139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.463188 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.463281 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:41.464023 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.464543 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.465479 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.465763 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.465876 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.467538 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:41.467544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:41.467546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:41.467574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:41.469415 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.469948 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.469950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.469952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.470031 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.470334 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.470339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.470341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.470565 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:41.470775 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.471020 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.471153 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":92,"threads_started":1}
I20260807 02:29:41.472539 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.458916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:38813 (local address 127.17.53.62:44331)
0807 02:29:41.459071 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.459076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.460047 (+ 971us) server_negotiation.cc:407] Connection header received
0807 02:29:41.461622 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.461629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.461755 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.461864 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:41.463205 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.463728 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.464935 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.465161 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.465283 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.468326 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:41.468330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:41.468333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:41.468380 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:41.470055 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.470698 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.470701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.470703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.470789 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:41.471235 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:41.471239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:41.471241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:41.471439 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:41.471519 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.472259 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.472379 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:29:41.473949 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.459388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:38813)
0807 02:29:41.459895 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.459937 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.461352 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.462083 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.462090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.462438 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:41.463044 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.463059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.463884 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.463887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.464728 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.464737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.465317 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.465321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.465498 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.466177 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:41.466206 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:41.468137 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:41.470213 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.470218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.470230 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.470600 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.470893 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:41.470896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:41.470897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:41.471092 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:41.471608 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:41.471614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:41.471955 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.472684 (+ 729us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.473043 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":115,"threads_started":1}
I20260807 02:29:41.475028 7352 heartbeater.cc:347] Connected to a master server at 127.17.53.62:44331
I20260807 02:29:41.475256 7352 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:41.475700 7352 heartbeater.cc:511] Master 127.17.53.62:44331 requested a full tablet report, sending...
I20260807 02:29:41.476655 6658 ts_manager.cc:194] Registered new tserver with Master: 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
I20260807 02:29:41.477222 6658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:38813
I20260807 02:29:41.485132 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:29:41.489427 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.485734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.17.53.62:44331)
0807 02:29:41.485859 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.485864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.486059 (+ 195us) server_negotiation.cc:407] Connection header received
0807 02:29:41.486237 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.486240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.486343 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.486444 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:29:41.486691 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.487178 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.488037 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.488262 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.488366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.488677 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:29:41.489069 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:29:41.489196 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.489259 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.489306 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:29:41.490022 6658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:29:41.493494 6658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:29:41.501271 6859 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 (DEFAULT_TABLE table=test-table [id=68c675a256444f7096ea4571e587f18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:41.501952 6859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8c7c0f04764f55ade44c37a44ec5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:41.504910 6793 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Bootstrap starting.
I20260807 02:29:41.505460 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.500073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40721 (local address 127.0.0.1:47298)
0807 02:29:41.500639 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.500654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.500829 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.501422 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.501425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.501444 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:41.501541 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.501547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.503164 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.503167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.504125 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.504134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.505071 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.505075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.505173 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.505199 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.505257 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.505330 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":12}
I20260807 02:29:41.505873 6793 tablet_bootstrap.cc:654] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:41.506151 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.500274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.17.53.2:40721)
0807 02:29:41.500642 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.500655 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.500834 (+ 179us) server_negotiation.cc:407] Connection header received
0807 02:29:41.500975 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.500982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.501249 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.501435 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:41.501654 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.503048 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.504255 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.504920 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.505639 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.505813 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.505875 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.505975 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":181,"threads_started":1}
I20260807 02:29:41.506824 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.500375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42681 (local address 127.0.0.1:36300)
0807 02:29:41.501032 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.501045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.501132 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.501975 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.501978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.501996 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:41.502082 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.502088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.503747 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.503753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.504916 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.504929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.506406 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.506409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.506511 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.506530 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.506623 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.506691 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":27,"thread_start_us":107,"threads_started":1}
I20260807 02:29:41.506902 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.500951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.17.53.3:42681)
0807 02:29:41.501441 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.501459 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.501480 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:29:41.501552 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.501557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.501837 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.501996 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:41.502202 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.503589 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.505060 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.506147 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.506287 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.506425 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.506655 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.506742 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":150,"threads_started":1}
I20260807 02:29:41.507484 6793 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: No bootstrap required, opened a new log
I20260807 02:29:41.507586 6793 ts_tablet_manager.cc:1397] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 02:29:41.508018 6793 raft_consensus.cc:348] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.508137 6793 raft_consensus.cc:374] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:41.508172 6793 raft_consensus.cc:729] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2390301cfc674befadd94e96df100198, State: Initialized, Role: FOLLOWER
I20260807 02:29:41.508315 6793 consensus_queue.cc:260] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.508599 6793 ts_tablet_manager.cc:1428] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 02:29:41.509649 7264 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 (DEFAULT_TABLE table=test-table [id=68c675a256444f7096ea4571e587f18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:41.509650 7064 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 (DEFAULT_TABLE table=test-table [id=68c675a256444f7096ea4571e587f18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:29:41.510720 7264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8c7c0f04764f55ade44c37a44ec5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:41.510766 7064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8c7c0f04764f55ade44c37a44ec5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:41.517449 7377 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Bootstrap starting.
I20260807 02:29:41.517449 7379 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Bootstrap starting.
I20260807 02:29:41.520179 7377 tablet_bootstrap.cc:654] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:41.520252 7379 tablet_bootstrap.cc:654] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:41.520999 7377 log.cc:824] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:41.521004 7379 log.cc:824] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:41.522782 7379 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: No bootstrap required, opened a new log
I20260807 02:29:41.522787 7377 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: No bootstrap required, opened a new log
I20260807 02:29:41.522979 7377 ts_tablet_manager.cc:1397] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:29:41.522985 7379 ts_tablet_manager.cc:1397] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:29:41.526114 7379 raft_consensus.cc:348] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.526114 7377 raft_consensus.cc:348] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.526328 7379 raft_consensus.cc:374] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:41.526352 7377 raft_consensus.cc:374] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:41.526392 7379 raft_consensus.cc:729] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74c5f83ddc84a28a22959390667ab8a, State: Initialized, Role: FOLLOWER
I20260807 02:29:41.526438 7377 raft_consensus.cc:729] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740dcc6befed431eb04ff6da18e7ede6, State: Initialized, Role: FOLLOWER
I20260807 02:29:41.526868 7377 consensus_queue.cc:260] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.526868 7379 consensus_queue.cc:260] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.527534 7352 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:41.527535 7164 heartbeater.cc:503] Master 127.17.53.62:44331 was elected leader, sending a full tablet report...
I20260807 02:29:41.527894 7377 ts_tablet_manager.cc:1428] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 02:29:41.527894 7379 ts_tablet_manager.cc:1428] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 02:29:41.566594 6957 tablet.cc:2369] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:29:41.708043 7354 tablet.cc:2369] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:29:41.724793 7382 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:41.725033 7382 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.726241 7382 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 2390301cfc674befadd94e96df100198 (127.17.53.1:44419)
I20260807 02:29:41.730582 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.726912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44419 (local address 127.17.53.3:51681)
0807 02:29:41.727309 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.727321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.727406 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.727699 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.727702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.727720 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:41.727839 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.727846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.728617 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.728620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.729428 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.729438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.730126 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.730133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.730243 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.730278 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.730347 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.730402 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":127,"threads_started":1}
I20260807 02:29:41.730623 7356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.726590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40721 (local address 127.17.53.3:55541)
0807 02:29:41.726741 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.726759 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:41.726868 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:41.727278 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:41.727281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:41.727310 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:29:41.727478 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.727488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.728421 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.728425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.729424 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:41.729435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.730209 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.730213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:41.730317 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.730365 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 02:29:41.730448 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 02:29:41.730514 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":44}
I20260807 02:29:41.730687 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.726728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:55541 (local address 127.17.53.2:40721)
0807 02:29:41.726899 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.726904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.726922 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:29:41.726988 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.726991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.727044 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.727156 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:41.727637 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.728308 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.729570 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.730059 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.730171 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.730237 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.730490 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.730563 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:29:41.730625 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.727072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:51681 (local address 127.17.53.1:44419)
0807 02:29:41.727320 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.727325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.727378 (+ 53us) server_negotiation.cc:407] Connection header received
0807 02:29:41.727518 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.727524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.727582 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.727669 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:29:41.727958 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.728505 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.729562 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.730000 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.730085 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.730172 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.730395 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.730487 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":64,"threads_started":1}
I20260807 02:29:41.731352 7099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8c7c0f04764f55ade44c37a44ec5a0" candidate_uuid: "740dcc6befed431eb04ff6da18e7ede6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74c5f83ddc84a28a22959390667ab8a" is_pre_election: true
I20260807 02:29:41.731376 6889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8c7c0f04764f55ade44c37a44ec5a0" candidate_uuid: "740dcc6befed431eb04ff6da18e7ede6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2390301cfc674befadd94e96df100198" is_pre_election: true
I20260807 02:29:41.731679 7099 raft_consensus.cc:2454] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 740dcc6befed431eb04ff6da18e7ede6 in term 0.
I20260807 02:29:41.731678 6889 raft_consensus.cc:2454] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 740dcc6befed431eb04ff6da18e7ede6 in term 0.
I20260807 02:29:41.732191 7196 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2390301cfc674befadd94e96df100198, 740dcc6befed431eb04ff6da18e7ede6; no voters:
I20260807 02:29:41.732496 7382 raft_consensus.cc:2781] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:29:41.732575 7382 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:29:41.732606 7382 raft_consensus.cc:3037] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:41.733938 7382 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.734345 7382 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [CANDIDATE]: Term 1 election: Requested vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 2390301cfc674befadd94e96df100198 (127.17.53.1:44419)
I20260807 02:29:41.734669 7099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8c7c0f04764f55ade44c37a44ec5a0" candidate_uuid: "740dcc6befed431eb04ff6da18e7ede6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74c5f83ddc84a28a22959390667ab8a"
I20260807 02:29:41.734740 6889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8c7c0f04764f55ade44c37a44ec5a0" candidate_uuid: "740dcc6befed431eb04ff6da18e7ede6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2390301cfc674befadd94e96df100198"
I20260807 02:29:41.734829 7099 raft_consensus.cc:3037] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:41.734896 6889 raft_consensus.cc:3037] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:41.735910 6889 raft_consensus.cc:2454] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 740dcc6befed431eb04ff6da18e7ede6 in term 1.
I20260807 02:29:41.736145 7099 raft_consensus.cc:2454] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 740dcc6befed431eb04ff6da18e7ede6 in term 1.
I20260807 02:29:41.736198 7196 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2390301cfc674befadd94e96df100198, 740dcc6befed431eb04ff6da18e7ede6; no voters:
I20260807 02:29:41.736356 7382 raft_consensus.cc:2781] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:41.736580 7382 raft_consensus.cc:686] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 LEADER]: Becoming Leader. State: Replica: 740dcc6befed431eb04ff6da18e7ede6, State: Running, Role: LEADER
I20260807 02:29:41.736954 7382 consensus_queue.cc:237] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:41.740289 6656 catalog_manager.cc:5495] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "740dcc6befed431eb04ff6da18e7ede6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } health_report { overall_health: UNKNOWN } } }
W20260807 02:29:41.770128 7165 tablet.cc:2369] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:29:41.800419 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.796649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.17.53.3:42681)
0807 02:29:41.796884 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:41.796888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:41.796903 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:29:41.797042 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:41.797044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:41.797108 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:41.797215 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:29:41.797515 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.798042 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.798933 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:41.799153 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:41.799241 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:41.799508 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:29:41.799958 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:29:41.800054 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:41.800140 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 02:29:41.800194 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 02:29:41.806803 6889 raft_consensus.cc:1261] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Refusing update from remote peer 740dcc6befed431eb04ff6da18e7ede6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:41.806814 7099 raft_consensus.cc:1261] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Refusing update from remote peer 740dcc6befed431eb04ff6da18e7ede6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:29:41.807560 7382 consensus_queue.cc:1035] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:41.807839 7388 consensus_queue.cc:1035] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:29:41.815059 7396 mvcc.cc:205] Tried to move back new op lower bound from 7315741826273026048 to 7315741825997611008. Current Snapshot: MvccSnapshot[applied={T|T < 7315741826273026048}]
I20260807 02:29:41.824365 6657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:41.824584 6657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:29:41.827492 6657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Sending DeleteTablet for 3 replicas of tablet 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:41.828369 7064 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:41 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:47298
I20260807 02:29:41.828514 6859 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:41 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:43710
I20260807 02:29:41.828613 7264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8c7c0f04764f55ade44c37a44ec5a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:29:41 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:36300
I20260807 02:29:41.829218 7410 tablet_replica.cc:330] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: stopping tablet replica
I20260807 02:29:41.829352 7408 tablet_replica.cc:330] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: stopping tablet replica
I20260807 02:29:41.829416 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 6727
I20260807 02:29:41.829516 7408 raft_consensus.cc:2229] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:41.829699 7408 raft_consensus.cc:2258] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:41.829980 7409 tablet_replica.cc:330] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: stopping tablet replica
I20260807 02:29:41.830174 7410 raft_consensus.cc:2229] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:29:41.830318 7409 raft_consensus.cc:2229] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:29:41.830649 7410 raft_consensus.cc:2258] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:41.830680 7409 raft_consensus.cc:2258] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:29:41.836313 7409 ts_tablet_manager.cc:1905] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:41.837080 7410 ts_tablet_manager.cc:1905] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:29:41.839282 7409 ts_tablet_manager.cc:1918] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:41.839399 7409 log.cc:1196] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:41.839804 7409 ts_tablet_manager.cc:1939] T 8a8c7c0f04764f55ade44c37a44ec5a0 P f74c5f83ddc84a28a22959390667ab8a: Deleting consensus metadata
I20260807 02:29:41.840107 7410 ts_tablet_manager.cc:1918] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:29:41.840253 7410 log.cc:1196] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:41.840699 7410 ts_tablet_manager.cc:1939] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 740dcc6befed431eb04ff6da18e7ede6: Deleting consensus metadata
I20260807 02:29:41.841372 6639 catalog_manager.cc:4841] TS f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): tablet 8a8c7c0f04764f55ade44c37a44ec5a0 (table test-table [id=68c675a256444f7096ea4571e587f18d]) successfully deleted
I20260807 02:29:41.842362 6638 catalog_manager.cc:4841] TS 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): tablet 8a8c7c0f04764f55ade44c37a44ec5a0 (table test-table [id=68c675a256444f7096ea4571e587f18d]) successfully deleted
W20260807 02:29:41.848239 6639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:44419 (error 108)
I20260807 02:29:41.848594 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 6963
I20260807 02:29:41.849709 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:41.849271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44419 (local address 127.0.0.1:43722)
0807 02:29:41.849508 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:41.849603 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260807 02:29:41.849917 6639 catalog_manager.cc:4584] TS 2390301cfc674befadd94e96df100198 (127.17.53.1:44419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a8c7c0f04764f55ade44c37a44ec5a0: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44419: connect: Connection refused (error 111)
I20260807 02:29:41.863495 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 7168
I20260807 02:29:41.873566 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 6612
I20260807 02:29:41.889364 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--webserver_interface=127.17.53.62
--webserver_port=34781
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:42.005852 7415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:42.006156 7415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:42.006242 7415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:42.010291 7415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:42.010386 7415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:42.010426 7415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:42.010490 7415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:42.010528 7415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:42.015894 7415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=34781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:42.017122 7415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:42.018203 7415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:42.027017 7422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.027014 7421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.027011 7424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:42.027441 7415 server_base.cc:1034] running on GCE node
I20260807 02:29:42.028183 7415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:42.029443 7415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:42.030637 7415 hybrid_clock.cc:648] HybridClock initialized: now 1786069782030617 us; error 31 us; skew 500 ppm
Aug 07 02:29:42 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:42.033563 7415 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:42.034530 7415 webserver.cc:460] Webserver started at http://127.17.53.62:34781/ using document root <none> and password file <none>
I20260807 02:29:42.035153 7415 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:42.035254 7415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:42.039284 7415 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:29:42.041524 7435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.042743 7415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:29:42.042901 7415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d507e2aa11324657992ebf3ed2dc72db"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "3023504dd4b5dab95024ddaf424924df"
server_key_iv: "b16244b40779a46a1ef963b36b3ddcfa"
server_key_version: "encryptionkey@0"
I20260807 02:29:42.043473 7415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:42.074826 7415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:42.078024 7415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:42.078269 7415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:42.085987 7415 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:44331
I20260807 02:29:42.086320 7515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:44331 every 8 connection(s)
I20260807 02:29:42.087141 7415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:42.091758 7517 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:29:42.093650 7517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap starting.
I20260807 02:29:42.097040 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 7415
I20260807 02:29:42.097581 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:44419
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=33991
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:42.114267 7517 log.cc:824] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:42.126456 7517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:29:42.126885 7517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db: Bootstrap complete.
I20260807 02:29:42.131208 7517 raft_consensus.cc:348] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:42.131601 7517 raft_consensus.cc:729] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Initialized, Role: FOLLOWER
I20260807 02:29:42.132107 7517 consensus_queue.cc:260] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:42.132225 7517 raft_consensus.cc:388] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:42.132314 7517 raft_consensus.cc:482] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:42.132439 7517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 2 FOLLOWER]: Advancing to term 3
I20260807 02:29:42.134543 7517 raft_consensus.cc:504] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:42.134892 7517 leader_election.cc:302] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d507e2aa11324657992ebf3ed2dc72db; no voters:
I20260807 02:29:42.135263 7517 leader_election.cc:288] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 02:29:42.135357 7522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 3 FOLLOWER]: Leader election won for term 3
I20260807 02:29:42.135569 7522 raft_consensus.cc:686] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [term 3 LEADER]: Becoming Leader. State: Replica: d507e2aa11324657992ebf3ed2dc72db, State: Running, Role: LEADER
I20260807 02:29:42.135867 7522 consensus_queue.cc:237] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } }
I20260807 02:29:42.136548 7517 sys_catalog.cc:564] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:42.137324 7523 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:42.137466 7523 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:42.138057 7524 sys_catalog.cc:455] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: SysCatalogTable state changed. Reason: New leader d507e2aa11324657992ebf3ed2dc72db. Latest consensus state: current_term: 3 leader_uuid: "d507e2aa11324657992ebf3ed2dc72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d507e2aa11324657992ebf3ed2dc72db" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 44331 } } }
I20260807 02:29:42.138212 7524 sys_catalog.cc:458] T 00000000000000000000000000000000 P d507e2aa11324657992ebf3ed2dc72db [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:42.138482 7533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:42.143736 7533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:42.144407 7533 catalog_manager.cc:1259] Loaded cluster ID: e1efa402e60f4117888c725225530dc3
I20260807 02:29:42.144469 7533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 02:29:42.145535 7533 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff9d35ab9ad google::LogMessage::Fail() at ??:0
@ 0x7ff9d35ae88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff9d35ab4de google::LogMessage::Flush() at ??:0
@ 0x7ff9d35aca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff9ddeeafdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff9ddeebd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff9ddee8579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff9ddf2c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff9d57fd3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff9d57fdc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff9d57ff3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff9d57ee512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff9d6e326db start_thread at ??:0
@ 0x7ff9d20a961f clone at ??:0
W20260807 02:29:42.235656 7520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:42.235939 7520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:42.236052 7520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:42.240377 7520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:42.240468 7520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:42.240564 7520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:42.246042 7520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:44419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=33991
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:42.247166 7520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:42.248255 7520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:42.258790 7574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.258790 7577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:42.259166 7520 server_base.cc:1034] running on GCE node
W20260807 02:29:42.258826 7573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:42.259613 7520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:42.260325 7520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:42.261505 7520 hybrid_clock.cc:648] HybridClock initialized: now 1786069782261484 us; error 39 us; skew 500 ppm
Aug 07 02:29:42 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:42.264528 7520 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:42.265650 7520 webserver.cc:460] Webserver started at http://127.17.53.1:33991/ using document root <none> and password file <none>
I20260807 02:29:42.266248 7520 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:42.266322 7520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:42.272136 7520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 02:29:42.276144 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.277642 7520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 02:29:42.277809 7520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "2390301cfc674befadd94e96df100198"
format_stamp: "Formatted at 2026-08-07 02:29:39 on dist-test-slave-g170"
server_key: "22847081187d5616b8f5495021d1bd30"
server_key_iv: "f1e4f5d79bd39526baef6f25631c22c5"
server_key_version: "encryptionkey@0"
I20260807 02:29:42.278427 7520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:42.311337 7520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:42.314916 7520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:42.315136 7520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:42.315840 7520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:42.317122 7600 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 02:29:42.321517 7520 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 02:29:42.321584 7520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 02:29:42.321635 7520 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 02:29:42.323361 7520 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 02:29:42.323421 7520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 02:29:42.323699 7600 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Bootstrap starting.
I20260807 02:29:42.335005 7520 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:44419
I20260807 02:29:42.335400 7767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:44419 every 8 connection(s)
I20260807 02:29:42.336405 7520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:42.346618 7600 log.cc:824] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:42.346753 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 7520
I20260807 02:29:42.347259 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:40721
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=32893
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:29:42.356436 7600 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:29:42.356958 7600 tablet_bootstrap.cc:492] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Bootstrap complete.
I20260807 02:29:42.358732 7600 ts_tablet_manager.cc:1397] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.008s
I20260807 02:29:42.362865 7600 raft_consensus.cc:348] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:42.363413 7600 raft_consensus.cc:729] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2390301cfc674befadd94e96df100198, State: Initialized, Role: FOLLOWER
I20260807 02:29:42.363965 7600 consensus_queue.cc:260] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:42.365082 7600 ts_tablet_manager.cc:1428] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260807 02:29:42.383844 7772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:29:42.338813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:36455)
0807 02:29:42.339465 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.339526 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:42.340568 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:42.383059 (+ 42491us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":186,"threads_started":1}
W20260807 02:29:42.384399 7770 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:44331 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:44331: Client connection negotiation failed: client connection to 127.17.53.62:44331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 02:29:42.385246 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.384976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:48725)
0807 02:29:42.385109 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.385189 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 02:29:42.386000 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.385776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:51727)
0807 02:29:42.385898 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.385950 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 02:29:42.386233 7770 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:29:42.473485 7774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:42.473770 7774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:42.473846 7774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:42.477995 7774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:42.478233 7774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:42.478405 7774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:42.483767 7774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:40721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=32893
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:42.484905 7774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:42.486243 7774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:42.496027 7786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.496055 7789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.496052 7785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:42.496356 7774 server_base.cc:1034] running on GCE node
I20260807 02:29:42.496860 7774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:42.497556 7774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:42.498759 7774 hybrid_clock.cc:648] HybridClock initialized: now 1786069782498724 us; error 40 us; skew 500 ppm
Aug 07 02:29:42 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:42.502485 7774 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:42.503661 7774 webserver.cc:460] Webserver started at http://127.17.53.2:32893/ using document root <none> and password file <none>
I20260807 02:29:42.504264 7774 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:42.504330 7774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:42.508427 7774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:42.510739 7799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.512243 7774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:29:42.512385 7774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "f74c5f83ddc84a28a22959390667ab8a"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "763cc559e54f9e5b4510253e1a599f65"
server_key_iv: "83bb3884b35ecc4c18acbb83dd69a931"
server_key_version: "encryptionkey@0"
I20260807 02:29:42.513026 7774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:42.551734 7774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:42.555248 7774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:42.555487 7774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:42.556180 7774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:42.557108 7774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:42.557158 7774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.557200 7774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:42.557216 7774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.567899 7774 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40721
I20260807 02:29:42.567930 7968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40721 every 8 connection(s)
I20260807 02:29:42.568991 7774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:42.573118 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.571311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:54137)
0807 02:29:42.571815 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.572288 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":108,"threads_started":1}
W20260807 02:29:42.573633 7969 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:44331 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:44331: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
I20260807 02:29:42.574219 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.573982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:35007)
0807 02:29:42.574104 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.574156 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 02:29:42.574924 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.574665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:36549)
0807 02:29:42.574807 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.574867 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 02:29:42.575104 7969 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 02:29:42.578276 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 7774
I20260807 02:29:42.578774 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:42681
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:44331
--webserver_port=35135
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:42.695237 7973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:42.695503 7973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:42.695547 7973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:42.699491 7973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:42.699538 7973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:42.699627 7973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:29:42.704739 7973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:42681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=35135
--tserver_master_addrs=127.17.53.62:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:42.705781 7973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:42.706810 7973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:42.716727 7980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.716709 7982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:42.716845 7986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:42.717116 7973 server_base.cc:1034] running on GCE node
I20260807 02:29:42.717605 7973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:42.718318 7973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:42.719523 7973 hybrid_clock.cc:648] HybridClock initialized: now 1786069782719488 us; error 46 us; skew 500 ppm
Aug 07 02:29:42 dist-test-slave-g170 krb5kdc[5787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:42.722648 7973 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:29:42.723596 7973 webserver.cc:460] Webserver started at http://127.17.53.3:35135/ using document root <none> and password file <none>
I20260807 02:29:42.724236 7973 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:42.724342 7973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:42.729171 7973 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:42.731554 7996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.733114 7973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:42.733278 7973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "740dcc6befed431eb04ff6da18e7ede6"
format_stamp: "Formatted at 2026-08-07 02:29:40 on dist-test-slave-g170"
server_key: "e47d6987ac200617b86d8f5230fa02a4"
server_key_iv: "16dbaefce2300999fe36f826616fa20a"
server_key_version: "encryptionkey@0"
I20260807 02:29:42.733901 7973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:42.747362 7973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:42.751341 7973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:42.751588 7973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:42.752332 7973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:42.753358 7973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:42.753443 7973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.753526 7973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:42.753575 7973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:42.764902 7973 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:42681
I20260807 02:29:42.764930 8168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:42681 every 8 connection(s)
I20260807 02:29:42.766109 7973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:29:42.767192 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 7973
I20260807 02:29:42.770908 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.768736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:40043)
0807 02:29:42.769342 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.769488 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":122,"threads_started":1}
W20260807 02:29:42.771540 8169 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:44331 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:44331: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
I20260807 02:29:42.772233 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.771980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:51767)
0807 02:29:42.772105 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.772162 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 02:29:42.773065 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:42.772825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:35871)
0807 02:29:42.772950 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:42.773013 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 02:29:42.773351 8169 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 02:29:43.387311 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.386857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:39483)
0807 02:29:43.387119 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:43.387176 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260807 02:29:43.576033 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.575619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:54101)
0807 02:29:43.575883 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:43.575936 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":71,"threads_started":1}
I20260807 02:29:43.774375 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.773937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:59221)
0807 02:29:43.774203 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:43.774269 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":72,"threads_started":1}
I20260807 02:29:43.793354 8178 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:43.793607 8178 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:43.795073 8178 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
Aug 07 02:29:43 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.2@KRBTEST.COM
Aug 07 02:29:43 dist-test-slave-g170 krb5kdc[5787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069782, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.3@KRBTEST.COM
I20260807 02:29:43.818444 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.795293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40721 (local address 127.17.53.1:44707)
0807 02:29:43.795521 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:43.795536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:43.795656 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:43.797612 (+ 1956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:43.797619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:43.798197 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:43.798713 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:43.798729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.800901 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.800905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:43.802499 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:43.802513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.803645 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.803650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:43.803968 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:43.810709 (+ 6741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:43.810745 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:43.812925 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:43.815421 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:43.815428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:43.815444 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:43.815796 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:43.816309 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:43.816313 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:43.816315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:43.816550 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:43.817522 (+ 972us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:43.817528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:43.817770 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 02:29:43.818000 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 02:29:43.818273 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":260,"spinlock_wait_cycles":35584}
I20260807 02:29:43.818862 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.795532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:44707 (local address 127.17.53.2:40721)
0807 02:29:43.795985 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:43.796018 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:43.796055 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:29:43.796858 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:43.796865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:43.797227 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:43.797537 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:43.798969 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.800703 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.802654 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.803474 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.803668 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:43.813231 (+ 9563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:43.813239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:43.813257 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:43.813289 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:43.815282 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:43.815986 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:43.815994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:43.816003 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:43.816175 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:43.816751 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:43.816754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:43.816756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:43.817411 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:43.817547 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:43.818136 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 02:29:43.818608 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":115,"threads_started":1}
I20260807 02:29:43.819947 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.795359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42681 (local address 127.17.53.1:32817)
0807 02:29:43.795782 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:43.795797 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:43.795907 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:43.797888 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:43.797891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:43.798196 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:43.798712 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:43.798729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.800901 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.800905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:43.802500 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:43.802513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.803645 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.803650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:43.803922 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:43.810709 (+ 6787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:43.810743 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:43.813304 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:43.816279 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:43.816282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:43.816284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:43.816564 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:43.817307 (+ 743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:43.817310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:43.817312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:43.817409 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:43.819340 (+ 1931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:43.819345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:43.819466 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 02:29:43.819587 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 02:29:43.819744 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":265,"spinlock_wait_cycles":3456,"thread_start_us":103,"threads_started":1}
I20260807 02:29:43.820150 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:43.795535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:32817 (local address 127.17.53.3:42681)
0807 02:29:43.795993 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:43.796016 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:43.796055 (+ 39us) server_negotiation.cc:407] Connection header received
0807 02:29:43.796853 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:43.796863 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:43.797227 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:43.797503 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:43.798931 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.800703 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.802654 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:43.803474 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:43.803668 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:43.813525 (+ 9857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:43.813531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:43.813544 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:43.813596 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:43.816141 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:43.816755 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:43.816761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:43.816769 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:43.817160 (+ 391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:43.818638 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:43.818641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:43.818643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:43.819184 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:43.819296 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:43.819717 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 02:29:43.819949 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":110,"threads_started":1}
W20260807 02:29:43.821755 7593 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
W20260807 02:29:43.822777 7591 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:43.822907 7591 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2390301cfc674befadd94e96df100198; no voters: 740dcc6befed431eb04ff6da18e7ede6, f74c5f83ddc84a28a22959390667ab8a
I20260807 02:29:43.823230 8178 raft_consensus.cc:2726] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:29:44.388835 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:44.388229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:47861)
0807 02:29:44.388574 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:44.388661 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":123,"threads_started":1}
I20260807 02:29:44.577310 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:44.576879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:55531)
0807 02:29:44.577161 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:44.577215 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
I20260807 02:29:44.775749 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:44.775262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:39311)
0807 02:29:44.775561 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:44.775623 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260807 02:29:45.390196 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:45.389787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:40357)
0807 02:29:45.390042 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:45.390097 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":80,"threads_started":1}
I20260807 02:29:45.468051 8190 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:45.468237 8190 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:45.468873 8190 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
W20260807 02:29:45.469839 7593 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
W20260807 02:29:45.470006 7591 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:45.470104 7591 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2390301cfc674befadd94e96df100198; no voters: 740dcc6befed431eb04ff6da18e7ede6, f74c5f83ddc84a28a22959390667ab8a
I20260807 02:29:45.470279 8190 raft_consensus.cc:2726] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:29:45.578521 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:45.578077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:45435)
0807 02:29:45.578344 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:45.578407 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260807 02:29:45.777313 8192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:45.776720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:48391)
0807 02:29:45.777143 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:45.777212 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":135,"threads_started":1}
I20260807 02:29:46.391446 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:46.391012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:45243)
0807 02:29:46.391292 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:46.391347 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260807 02:29:46.579668 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:46.579293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:36709)
0807 02:29:46.579527 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:46.579577 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260807 02:29:46.778683 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:46.778223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:59571)
0807 02:29:46.778526 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:46.778587 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":78,"threads_started":1}
I20260807 02:29:47.392854 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:47.392332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:59771)
0807 02:29:47.392632 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:47.392685 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":92,"threads_started":1}
I20260807 02:29:47.580896 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:47.580451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:40803)
0807 02:29:47.580694 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:47.580751 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260807 02:29:47.780098 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:47.779626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:55567)
0807 02:29:47.779903 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:47.779967 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260807 02:29:48.394025 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:48.393598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:47399)
0807 02:29:48.393869 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:48.393924 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":84,"threads_started":1}
I20260807 02:29:48.582085 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:48.581670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:47455)
0807 02:29:48.581935 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:48.581990 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":96,"threads_started":1}
I20260807 02:29:48.781314 8204 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:48.781505 8204 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:48.781795 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:48.781284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:60247)
0807 02:29:48.781550 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:48.781620 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
I20260807 02:29:48.782121 8204 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
W20260807 02:29:48.782971 7593 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
W20260807 02:29:48.783123 7591 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:48.783222 7591 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2390301cfc674befadd94e96df100198; no voters: 740dcc6befed431eb04ff6da18e7ede6, f74c5f83ddc84a28a22959390667ab8a
I20260807 02:29:48.783413 8204 raft_consensus.cc:2726] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:29:49.395364 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:49.394923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:33721)
0807 02:29:49.395187 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:49.395244 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260807 02:29:49.583595 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:49.582938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:54219)
0807 02:29:49.583326 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:49.583448 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
I20260807 02:29:49.783279 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:49.782685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:59801)
0807 02:29:49.783073 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:49.783175 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":142,"threads_started":1}
I20260807 02:29:50.396696 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:50.396199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:33591)
0807 02:29:50.396521 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:50.396585 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
I20260807 02:29:50.584937 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:50.584433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:54169)
0807 02:29:50.584695 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:50.584757 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260807 02:29:50.784672 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:50.784211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:59919)
0807 02:29:50.784521 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:50.784575 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":126,"threads_started":1}
I20260807 02:29:51.307983 8218 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:51.308154 8218 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:51.308595 8218 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
W20260807 02:29:51.309453 7593 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
W20260807 02:29:51.309562 7591 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:51.309664 7591 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2390301cfc674befadd94e96df100198; no voters: 740dcc6befed431eb04ff6da18e7ede6, f74c5f83ddc84a28a22959390667ab8a
I20260807 02:29:51.309808 8218 raft_consensus.cc:2726] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:29:51.398123 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:51.397658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:42445)
0807 02:29:51.397953 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:51.398017 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":86,"threads_started":1}
I20260807 02:29:51.586500 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:51.586015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:46321)
0807 02:29:51.586339 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:51.586403 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
I20260807 02:29:51.785914 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:51.785441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:54773)
0807 02:29:51.785714 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:51.785814 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260807 02:29:52.399696 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:52.399209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:43099)
0807 02:29:52.399532 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:52.399590 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":119,"threads_started":1}
I20260807 02:29:52.587853 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:52.587439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:56413)
0807 02:29:52.587689 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:52.587755 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260807 02:29:52.787305 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:52.786783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:51171)
0807 02:29:52.787096 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:52.787174 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260807 02:29:53.401057 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:53.400574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:48419)
0807 02:29:53.400903 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:53.400959 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260807 02:29:53.589211 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:53.588691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:41703)
0807 02:29:53.589049 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:53.589110 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":137,"threads_started":1}
I20260807 02:29:53.788515 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:53.788061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:38255)
0807 02:29:53.788348 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:53.788410 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260807 02:29:54.402460 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:54.401941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:43751)
0807 02:29:54.402283 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:54.402354 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
I20260807 02:29:54.590477 8234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:54.590037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:47807)
0807 02:29:54.590305 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:54.590370 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
I20260807 02:29:54.789850 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:54.789433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:36835)
0807 02:29:54.789691 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:54.789747 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
I20260807 02:29:55.403759 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:55.403286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:47061)
0807 02:29:55.403554 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:55.403612 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
I20260807 02:29:55.525074 8240 raft_consensus.cc:482] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:29:55.525266 8240 raft_consensus.cc:504] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74c5f83ddc84a28a22959390667ab8a" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40721 } } peers { permanent_uuid: "740dcc6befed431eb04ff6da18e7ede6" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42681 } } peers { permanent_uuid: "2390301cfc674befadd94e96df100198" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44419 } }
I20260807 02:29:55.525750 8240 leader_election.cc:288] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721), 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681)
W20260807 02:29:55.526723 7591 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 740dcc6befed431eb04ff6da18e7ede6 (127.17.53.3:42681): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
W20260807 02:29:55.526825 7593 leader_election.cc:341] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74c5f83ddc84a28a22959390667ab8a (127.17.53.2:40721): Not found: Tablet not found: 8a8c7c0f04764f55ade44c37a44ec5a0
I20260807 02:29:55.526942 7593 leader_election.cc:302] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2390301cfc674befadd94e96df100198; no voters: 740dcc6befed431eb04ff6da18e7ede6, f74c5f83ddc84a28a22959390667ab8a
I20260807 02:29:55.527184 8240 raft_consensus.cc:2726] T 8a8c7c0f04764f55ade44c37a44ec5a0 P 2390301cfc674befadd94e96df100198 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:29:55.591789 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:55.591326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:55231)
0807 02:29:55.591603 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:55.591679 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260807 02:29:55.791074 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:55.790607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:47929)
0807 02:29:55.790909 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:55.790966 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":95,"threads_started":1}
I20260807 02:29:56.405084 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:56.404610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:43123)
0807 02:29:56.404927 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:56.404982 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":112,"threads_started":1}
I20260807 02:29:56.593183 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:56.592615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:41621)
0807 02:29:56.592983 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:56.593058 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":82,"threads_started":1}
I20260807 02:29:56.792423 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:56.791933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:39957)
0807 02:29:56.792214 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:56.792272 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260807 02:29:57.406461 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:57.406043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.1:56727)
0807 02:29:57.406306 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:57.406361 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260807 02:29:57.594493 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:57.594054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.2:55041)
0807 02:29:57.594333 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:57.594390 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260807 02:29:57.769865 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 7520
I20260807 02:29:57.786463 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 7774
I20260807 02:29:57.794190 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:57.793623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:44331 (local address 127.17.53.3:33197)
0807 02:29:57.793958 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:57.794039 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":80,"threads_started":1}
I20260807 02:29:57.799161 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 7973
2026-08-07T02:29:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19857 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:29:57 dist-test-slave-g170 krb5kdc[8254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:29:57 dist-test-slave-g170 krb5kdc[8254](info): set up 2 sockets
Aug 07 02:29:57 dist-test-slave-g170 krb5kdc[8254](info): commencing operation
krb5kdc: starting...
W20260807 02:29:59.062050 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:29:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:29:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:29:59.193272 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:59.325323 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:59.325567 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:59.325605 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:59.329551 8274 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 02:29:59.329596 8274 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 02:29:59.329612 8274 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 02:29:59.329627 8274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:29:59.329643 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:59.329685 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:59.329731 8274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:29:59.329770 8274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:29:59.334846 8274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:59.335813 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:59.336823 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:59.345613 8281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:59.345659 8284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.345693 8274 server_base.cc:1034] running on GCE node
W20260807 02:29:59.345747 8286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.346455 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:59.347486 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:59.348649 8274 hybrid_clock.cc:648] HybridClock initialized: now 1786069799348647 us; error 44 us; skew 500 ppm
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:59.351455 8274 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:29:59.352308 8274 webserver.cc:460] Webserver started at http://127.17.53.62:33671/ using document root <none> and password file <none>
I20260807 02:29:59.352927 8274 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:59.353024 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:59.353271 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:59.355289 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "f729d9b61fbb442aa25d2e3a0a177066"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "2d4d16d04e5d6341a73b8f238adc87fb"
server_key_iv: "97dd18fcda5105ba55508589a2f5977b"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.355870 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "f729d9b61fbb442aa25d2e3a0a177066"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "2d4d16d04e5d6341a73b8f238adc87fb"
server_key_iv: "97dd18fcda5105ba55508589a2f5977b"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.359637 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 02:29:59.362076 8301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.363323 8274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:29:59.363466 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "f729d9b61fbb442aa25d2e3a0a177066"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "2d4d16d04e5d6341a73b8f238adc87fb"
server_key_iv: "97dd18fcda5105ba55508589a2f5977b"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.363575 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:59.381556 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:59.385003 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:59.385213 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:59.392546 8274 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35283
I20260807 02:29:59.392897 8386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35283 every 8 connection(s)
I20260807 02:29:59.393586 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:29:59.396529 8387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:29:59.400992 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 8274
I20260807 02:29:59.401124 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:29:59.401423 17620 external_mini_cluster.cc:1428] Setting key 07673cfa6477496b8d11a509a0f6add1
I20260807 02:29:59.402974 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap starting.
I20260807 02:29:59.405527 8387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Neither blocks nor log segments found. Creating new log.
I20260807 02:29:59.406351 8387 log.cc:824] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Log is configured to *not* fsync() on all Append() calls
I20260807 02:29:59.408233 8387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: No bootstrap required, opened a new log
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:59.412861 8387 raft_consensus.cc:348] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:29:59.413141 8387 raft_consensus.cc:374] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:29:59.413208 8387 raft_consensus.cc:729] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Initialized, Role: FOLLOWER
I20260807 02:29:59.413697 8387 consensus_queue.cc:260] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:29:59.413868 8387 raft_consensus.cc:388] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:29:59.413944 8387 raft_consensus.cc:482] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:29:59.414047 8387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:29:59.415246 8387 raft_consensus.cc:504] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:29:59.415606 8387 leader_election.cc:302] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f729d9b61fbb442aa25d2e3a0a177066; no voters:
I20260807 02:29:59.415943 8387 leader_election.cc:288] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:29:59.416074 8396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:29:59.416325 8396 raft_consensus.cc:686] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 LEADER]: Becoming Leader. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Running, Role: LEADER
I20260807 02:29:59.416666 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:59.402866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.17.53.62:35283)
0807 02:29:59.403304 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:59.403320 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:59.403356 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:29:59.404040 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:59.404063 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:59.404385 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:59.404660 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:59.405750 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.406831 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.407739 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.408216 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.408396 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:59.411203 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:59.411212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:59.411245 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:59.411288 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:59.413363 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.413864 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.413871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.413878 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.414026 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.414359 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.414363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.414364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.414860 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:59.415072 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:59.415314 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 02:29:59.415619 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":139,"threads_started":1}
I20260807 02:29:59.416713 8396 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:29:59.417271 8387 sys_catalog.cc:564] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:29:59.417990 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729d9b61fbb442aa25d2e3a0a177066. Latest consensus state: current_term: 1 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:29:59.418138 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:59.418936 8407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:29:59.419531 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:29:59.419716 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:29:59.422226 8407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:29:59.428223 8407 catalog_manager.cc:1347] Generated new cluster ID: 261790c262ff46feaa5dfa8fb9d6da0d
I20260807 02:29:59.428319 8407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:29:59.455614 8407 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:29:59.456928 8407 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:29:59.463871 8407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Generated new TSK 0
I20260807 02:29:59.464609 8407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:29:59.500598 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:59.623404 8430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:59.623690 8430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:59.623739 8430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:59.627684 8430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:59.627746 8430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:59.627806 8430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:29:59.633235 8430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:59.634315 8430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:59.635370 8430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:59.645499 8441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.645699 8430 server_base.cc:1034] running on GCE node
W20260807 02:29:59.645520 8443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:59.645499 8439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.646450 8430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:59.647137 8430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:59.648314 8430 hybrid_clock.cc:648] HybridClock initialized: now 1786069799648283 us; error 42 us; skew 500 ppm
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:59.654039 8430 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:29:59.655078 8430 webserver.cc:460] Webserver started at http://127.17.53.1:42215/ using document root <none> and password file <none>
I20260807 02:29:59.655726 8430 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:59.655829 8430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:59.656083 8430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:59.658313 8430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "9ea24f3a43c44a364a99a92302d0e480"
server_key_iv: "3a42ee8799df6c2098dc9f31bf2e6709"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.658936 8430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "9ea24f3a43c44a364a99a92302d0e480"
server_key_iv: "3a42ee8799df6c2098dc9f31bf2e6709"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.662997 8430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:29:59.665748 8454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.667091 8430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:29:59.667270 8430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "9ea24f3a43c44a364a99a92302d0e480"
server_key_iv: "3a42ee8799df6c2098dc9f31bf2e6709"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.667397 8430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:59.684181 8430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:59.687383 8430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:59.687608 8430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:59.688238 8430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:59.689177 8430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:59.689254 8430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.689325 8430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:59.689375 8430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.699903 8430 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:41617
I20260807 02:29:59.699930 8638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:41617 every 8 connection(s)
I20260807 02:29:59.701126 8430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:29:59.710026 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 8430
I20260807 02:29:59.710201 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:29:59.710542 17620 external_mini_cluster.cc:1428] Setting key b488651069ee601c60b3830928faceaa
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:59.716403 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:59.703161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:38061 (local address 127.17.53.62:35283)
0807 02:29:59.703309 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:59.703313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:59.704188 (+ 875us) server_negotiation.cc:407] Connection header received
0807 02:29:59.705695 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:59.705699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:59.705762 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:59.705863 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:59.707073 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.707631 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.708881 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.709117 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.709232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:59.712035 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:59.712041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:59.712046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:59.712077 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:59.713854 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.714541 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.714548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.714552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.714678 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.715069 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.715074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.715079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.715351 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:59.715475 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:59.715984 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 02:29:59.716136 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:29:59.717250 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:59.703515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:38061)
0807 02:29:59.704019 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:59.704071 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:59.705470 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:59.706049 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:59.706056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:59.706419 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:59.706913 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:59.706926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.707777 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.707782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:59.708674 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:59.708684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.709281 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.709286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:59.709465 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:59.710128 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:59.710156 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:59.711816 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:59.713979 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:59.713988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:59.714008 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:59.714412 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:59.714816 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:59.714818 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:59.714820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:59.714967 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:59.715465 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:59.715470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:59.715702 (+ 232us) client_negotiation.cc:769] Sending connection context
0807 02:29:59.715890 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 02:29:59.716150 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":145,"threads_started":1}
I20260807 02:29:59.719267 8640 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:29:59.719514 8640 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:59.720041 8640 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
I20260807 02:29:59.721931 8331 ts_manager.cc:194] Registered new tserver with Master: 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617)
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:29:59.723584 8331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:38061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:29:59.744458 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:29:59.865819 8648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:29:59.866144 8648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:29:59.866223 8648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:29:59.870651 8648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:29:59.870754 8648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:29:59.870848 8648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:29:59.876674 8648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:29:59.877969 8648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:29:59.879082 8648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:29:59.889451 8658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:29:59.889499 8663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.889549 8648 server_base.cc:1034] running on GCE node
W20260807 02:29:59.889457 8660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:29:59.890374 8648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:29:59.891055 8648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:29:59.892258 8648 hybrid_clock.cc:648] HybridClock initialized: now 1786069799892219 us; error 52 us; skew 500 ppm
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:29:59.895517 8648 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:29:59.896468 8648 webserver.cc:460] Webserver started at http://127.17.53.2:37033/ using document root <none> and password file <none>
I20260807 02:29:59.897117 8648 fs_manager.cc:359] Metadata directory not provided
I20260807 02:29:59.897189 8648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:29:59.897374 8648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:29:59.899537 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "4a318aeac5534af280f8050cbf9f3eb6"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "df628ad028d360599e35b2bd2bd59f4a"
server_key_iv: "a87934c494f43416a60f2a4b534b7191"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.900110 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "4a318aeac5534af280f8050cbf9f3eb6"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "df628ad028d360599e35b2bd2bd59f4a"
server_key_iv: "a87934c494f43416a60f2a4b534b7191"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.904131 8648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:29:59.906703 8675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.908075 8648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 02:29:59.908217 8648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "4a318aeac5534af280f8050cbf9f3eb6"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "df628ad028d360599e35b2bd2bd59f4a"
server_key_iv: "a87934c494f43416a60f2a4b534b7191"
server_key_version: "encryptionkey@0"
I20260807 02:29:59.908331 8648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:29:59.932456 8648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:29:59.935650 8648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:29:59.935858 8648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:29:59.936503 8648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:29:59.937430 8648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:29:59.937479 8648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.937548 8648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:29:59.937580 8648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:29:59.946827 8648 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39377
I20260807 02:29:59.946906 8844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39377 every 8 connection(s)
I20260807 02:29:59.947767 8648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:29:59.952612 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 8648
I20260807 02:29:59.952791 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:29:59.953125 17620 external_mini_cluster.cc:1428] Setting key f548a0fa02f94a73b41f989701ffb560
Aug 07 02:29:59 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:29:59.964875 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:59.950096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:40253 (local address 127.17.53.62:35283)
0807 02:29:59.950267 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 02:29:59.950271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:29:59.951111 (+ 840us) server_negotiation.cc:407] Connection header received
0807 02:29:59.952536 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:29:59.952539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:29:59.952589 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:29:59.952681 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:29:59.954126 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.954901 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.956636 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.957002 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.957130 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:59.960149 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:29:59.960156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:29:59.960163 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:29:59.960202 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:29:59.962255 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.962825 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.962831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.962835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.962937 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:29:59.963421 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:29:59.963425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:29:59.963427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:29:59.963735 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:29:59.963851 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:29:59.964454 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 02:29:59.964609 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 02:29:59.965672 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:29:59.950466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:40253)
0807 02:29:59.950954 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 02:29:59.950996 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:29:59.952256 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:29:59.952920 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:29:59.952931 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:29:59.953443 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:29:59.953986 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:59.954001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.955039 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.955043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:59.956489 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:29:59.956504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:29:59.957179 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:29:59.957184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:29:59.957377 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:29:59.958112 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:29:59.958131 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:29:59.959929 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:29:59.962402 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:59.962407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:59.962420 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:59.962704 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:59.963078 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:29:59.963081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:29:59.963083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:29:59.963306 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:29:59.963868 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:29:59.963874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:29:59.964161 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 02:29:59.964356 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 02:29:59.964633 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:29:59.966961 8847 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:29:59.967308 8847 heartbeater.cc:464] Registering TS with master...
I20260807 02:29:59.967898 8847 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:29:59.969172 8331 ts_manager.cc:194] Registered new tserver with Master: 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
I20260807 02:29:59.970101 8331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:40253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:29:59.988008 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:00.108577 8855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:00.108945 8855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:00.109042 8855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:00.113119 8855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:00.113204 8855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:00.113292 8855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:30:00.118861 8855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:00.120007 8855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:00.121165 8855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:00.131451 8866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:00.131770 8855 server_base.cc:1034] running on GCE node
W20260807 02:30:00.131511 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:00.131454 8865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:00.132340 8855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:00.133071 8855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:00.134245 8855 hybrid_clock.cc:648] HybridClock initialized: now 1786069800134224 us; error 35 us; skew 500 ppm
Aug 07 02:30:00 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069800, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:00.137251 8855 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:30:00.138135 8855 webserver.cc:460] Webserver started at http://127.17.53.3:40101/ using document root <none> and password file <none>
I20260807 02:30:00.138742 8855 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:00.138873 8855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:00.139123 8855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:30:00.141284 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "f61d76f543f24d0e83086f9f2996e3e7"
format_stamp: "Formatted at 2026-08-07 02:30:00 on dist-test-slave-g170"
server_key: "edf3536ac13d2fcc44aef4b16da9aa5e"
server_key_iv: "7f3f19b3556632aec8c6df69d9d0d74c"
server_key_version: "encryptionkey@0"
I20260807 02:30:00.141867 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "f61d76f543f24d0e83086f9f2996e3e7"
format_stamp: "Formatted at 2026-08-07 02:30:00 on dist-test-slave-g170"
server_key: "edf3536ac13d2fcc44aef4b16da9aa5e"
server_key_iv: "7f3f19b3556632aec8c6df69d9d0d74c"
server_key_version: "encryptionkey@0"
I20260807 02:30:00.145795 8855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:30:00.148379 8879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:00.150065 8855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:00.150261 8855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "f61d76f543f24d0e83086f9f2996e3e7"
format_stamp: "Formatted at 2026-08-07 02:30:00 on dist-test-slave-g170"
server_key: "edf3536ac13d2fcc44aef4b16da9aa5e"
server_key_iv: "7f3f19b3556632aec8c6df69d9d0d74c"
server_key_version: "encryptionkey@0"
I20260807 02:30:00.150396 8855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:00.182258 8855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:00.185583 8855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:00.185803 8855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:00.186455 8855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:00.187373 8855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:00.187448 8855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:00.187525 8855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:00.187577 8855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:00.197296 8855 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:45791
I20260807 02:30:00.197320 9046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:45791 every 8 connection(s)
I20260807 02:30:00.198328 8855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:30:00.206025 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 8855
I20260807 02:30:00.206158 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:30:00.206508 17620 external_mini_cluster.cc:1428] Setting key c7d97940eb1705e66e84de9b47838074
Aug 07 02:30:00 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069800, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:30:00.212625 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.200358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:51045 (local address 127.17.53.62:35283)
0807 02:30:00.200536 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.200541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.201478 (+ 937us) server_negotiation.cc:407] Connection header received
0807 02:30:00.202907 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.202910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.202958 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.203037 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:00.204226 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.204750 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.205897 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.206273 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.206397 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.209382 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:00.209385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:00.209387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:00.209410 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:00.210701 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:00.211163 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:00.211165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:00.211167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:00.211255 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:00.211618 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:00.211621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:00.211622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:00.211800 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:00.211881 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.212335 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.212474 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:30:00.213426 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.200718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:51045)
0807 02:30:00.201291 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.201353 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.202678 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.203196 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.203205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.203553 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:00.204086 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.204101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.204863 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.204866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.205752 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.205762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.206428 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.206431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.206613 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.207330 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:00.207368 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:00.209238 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:00.210808 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:00.210813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:00.210823 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:00.211076 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:00.211378 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:00.211385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:00.211387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:00.211533 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:00.211894 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:00.211899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:00.212118 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.212280 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.212585 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":120,"threads_started":1}
I20260807 02:30:00.214540 9047 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:30:00.214792 9047 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:00.215273 9047 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
I20260807 02:30:00.216289 8331 ts_manager.cc:194] Registered new tserver with Master: f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791)
I20260807 02:30:00.216934 8331 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:51045
I20260807 02:30:00.220943 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:30:00.230427 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.222963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.17.53.62:35283)
0807 02:30:00.223086 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.223090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.223309 (+ 219us) server_negotiation.cc:407] Connection header received
0807 02:30:00.223383 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.223386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.223428 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.223496 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:00.224214 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.224711 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.225633 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.225858 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.225949 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.227325 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:00.227328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:00.227330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:00.227352 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:00.228747 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:00.229228 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:00.229231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:00.229232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:00.229328 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:00.229625 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:00.229628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:00.229629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:00.229867 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:00.229987 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.230198 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.230313 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:30:00.233848 8331 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:30:00.236985 8331 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:30:00.251967 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.246850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.17.53.2:39377)
0807 02:30:00.247231 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.247242 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.247493 (+ 251us) server_negotiation.cc:407] Connection header received
0807 02:30:00.247581 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.247587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.247774 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.247906 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:00.248146 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.249335 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.250544 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251189 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.251315 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.251476 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.251690 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.251804 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":158,"threads_started":1}
I20260807 02:30:00.251973 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.245482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41617 (local address 127.0.0.1:49328)
0807 02:30:00.245844 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.245896 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.246048 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.247467 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.247469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.247482 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:00.247649 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.247656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.248834 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.248837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.250275 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.250287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251341 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.251345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.251477 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.251528 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.251631 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.251797 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":73,"thread_start_us":75,"threads_started":1}
I20260807 02:30:00.252135 9069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.245685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:45791 (local address 127.0.0.1:58992)
0807 02:30:00.246435 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.246452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.246747 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.247380 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.247389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.247421 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:00.247645 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.247655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.249499 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.249502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.250695 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.250705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251790 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.251794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.251878 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.251900 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.251948 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.252017 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":54,"thread_start_us":89,"threads_started":1}
I20260807 02:30:00.252712 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.246204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.0.0.1:54340)
0807 02:30:00.247294 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.247310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.247396 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.247914 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.247917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.247936 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:00.248012 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.248018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.249483 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.249487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.250411 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.250422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251317 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.251320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.251481 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.251535 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.251651 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.251819 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":157,"thread_start_us":105,"threads_started":1}
I20260807 02:30:00.252744 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.246535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.17.53.1:41617)
0807 02:30:00.246972 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.246977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.246996 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:30:00.247091 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.247097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.247300 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.247578 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:00.247757 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.248645 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.250423 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251182 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.252128 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.252453 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.252506 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.252592 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":127,"threads_started":1}
I20260807 02:30:00.252749 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.246328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.17.53.3:45791)
0807 02:30:00.246798 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.246812 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.246837 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:30:00.246929 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.246934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.247158 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.247374 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:00.248066 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.249320 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.250829 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.251642 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.252310 (+ 668us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.252452 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.252506 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.252592 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":101,"threads_started":1}
I20260807 02:30:00.255606 8952 tablet_service.cc:1467] Processing CreateTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 (DEFAULT_TABLE table=test-table [id=93ffa7db973c4fc7a9097307ed63422b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:00.255606 8752 tablet_service.cc:1467] Processing CreateTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 (DEFAULT_TABLE table=test-table [id=93ffa7db973c4fc7a9097307ed63422b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:00.255561 8535 tablet_service.cc:1467] Processing CreateTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 (DEFAULT_TABLE table=test-table [id=93ffa7db973c4fc7a9097307ed63422b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:00.256701 8535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578414ce5f374aa3ae859d79618a3fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:00.256711 8752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578414ce5f374aa3ae859d79618a3fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:00.256723 8952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578414ce5f374aa3ae859d79618a3fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:00.262964 9077 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap starting.
I20260807 02:30:00.264474 9082 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Bootstrap starting.
I20260807 02:30:00.265056 9079 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Bootstrap starting.
I20260807 02:30:00.265663 9077 tablet_bootstrap.cc:654] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:00.266911 9077 log.cc:824] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:00.267199 9079 tablet_bootstrap.cc:654] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:00.267235 9082 tablet_bootstrap.cc:654] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:00.268165 9082 log.cc:824] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:00.268688 9079 log.cc:824] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:00.270566 9077 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: No bootstrap required, opened a new log
I20260807 02:30:00.270810 9077 ts_tablet_manager.cc:1397] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 02:30:00.273028 9079 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: No bootstrap required, opened a new log
I20260807 02:30:00.273259 9079 ts_tablet_manager.cc:1397] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260807 02:30:00.277093 9079 raft_consensus.cc:348] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.277377 9079 raft_consensus.cc:374] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:00.277441 9079 raft_consensus.cc:729] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f61d76f543f24d0e83086f9f2996e3e7, State: Initialized, Role: FOLLOWER
I20260807 02:30:00.277858 9079 consensus_queue.cc:260] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.278518 9047 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
I20260807 02:30:00.278891 9079 ts_tablet_manager.cc:1428] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 02:30:00.278977 9077 raft_consensus.cc:348] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.279299 9077 raft_consensus.cc:374] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:00.279366 9077 raft_consensus.cc:729] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09eaae16e56d4a40b1a4d55d6a62a3f8, State: Initialized, Role: FOLLOWER
I20260807 02:30:00.279809 9077 consensus_queue.cc:260] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.280479 8640 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
I20260807 02:30:00.280962 9082 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: No bootstrap required, opened a new log
I20260807 02:30:00.281198 9082 ts_tablet_manager.cc:1397] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260807 02:30:00.281919 9077 ts_tablet_manager.cc:1428] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent starting tablet: real 0.011s user 0.002s sys 0.006s
I20260807 02:30:00.285583 9082 raft_consensus.cc:348] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.285961 9082 raft_consensus.cc:374] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:00.286046 9082 raft_consensus.cc:729] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a318aeac5534af280f8050cbf9f3eb6, State: Initialized, Role: FOLLOWER
I20260807 02:30:00.286562 9082 consensus_queue.cc:260] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.287518 8847 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
I20260807 02:30:00.287778 9082 ts_tablet_manager.cc:1428] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:30:00.301874 9089 raft_consensus.cc:482] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:00.302090 9089 raft_consensus.cc:504] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.303241 9089 leader_election.cc:288] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
I20260807 02:30:00.308444 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.303584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:45791 (local address 127.17.53.1:40765)
0807 02:30:00.304031 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.304044 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.304181 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.304505 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.304508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.304538 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:00.304854 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.304864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.306077 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.306080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.306998 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.307009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.307987 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.307990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.308076 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.308100 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.308167 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.308250 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":96,"spinlock_wait_cycles":34560,"thread_start_us":117,"threads_started":1}
I20260807 02:30:00.308435 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.303664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:40765 (local address 127.17.53.3:45791)
0807 02:30:00.303792 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.303797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.304148 (+ 351us) server_negotiation.cc:407] Connection header received
0807 02:30:00.304332 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.304336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.304390 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.304478 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:00.304998 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.305884 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.307151 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.307851 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.307972 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.308042 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.308214 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.308302 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:30:00.309319 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578414ce5f374aa3ae859d79618a3fc0" candidate_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61d76f543f24d0e83086f9f2996e3e7" is_pre_election: true
I20260807 02:30:00.309677 8980 raft_consensus.cc:2454] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09eaae16e56d4a40b1a4d55d6a62a3f8 in term 0.
I20260807 02:30:00.310081 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.303584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.17.53.1:51811)
0807 02:30:00.304015 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.304030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:00.304181 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:00.304505 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:00.304508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:00.304538 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:00.304854 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.304864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.307173 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.307176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.308451 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:00.308467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.309571 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.309575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:00.309679 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.309716 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 02:30:00.309822 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 02:30:00.309900 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":59,"spinlock_wait_cycles":15104,"thread_start_us":175,"threads_started":1}
I20260807 02:30:00.310109 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.303665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:51811 (local address 127.17.53.2:39377)
0807 02:30:00.303794 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.303798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.304147 (+ 349us) server_negotiation.cc:407] Connection header received
0807 02:30:00.304315 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.304319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.304379 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.304480 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:00.304998 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.305882 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.308620 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.309293 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.309388 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.309455 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.309878 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.309955 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 02:30:00.310261 8461 leader_election.cc:302] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8, f61d76f543f24d0e83086f9f2996e3e7; no voters:
I20260807 02:30:00.310613 9089 raft_consensus.cc:2781] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:30:00.310714 9089 raft_consensus.cc:482] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:30:00.310708 8779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578414ce5f374aa3ae859d79618a3fc0" candidate_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a318aeac5534af280f8050cbf9f3eb6" is_pre_election: true
I20260807 02:30:00.310808 9089 raft_consensus.cc:3037] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:00.310958 8779 raft_consensus.cc:2454] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09eaae16e56d4a40b1a4d55d6a62a3f8 in term 0.
I20260807 02:30:00.312116 9089 raft_consensus.cc:504] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.312510 9089 leader_election.cc:288] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 1 election: Requested vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
I20260807 02:30:00.312858 8779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578414ce5f374aa3ae859d79618a3fc0" candidate_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a318aeac5534af280f8050cbf9f3eb6"
I20260807 02:30:00.312906 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578414ce5f374aa3ae859d79618a3fc0" candidate_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61d76f543f24d0e83086f9f2996e3e7"
I20260807 02:30:00.312978 8779 raft_consensus.cc:3037] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:00.313077 8980 raft_consensus.cc:3037] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:00.314066 8779 raft_consensus.cc:2454] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09eaae16e56d4a40b1a4d55d6a62a3f8 in term 1.
I20260807 02:30:00.314333 8980 raft_consensus.cc:2454] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09eaae16e56d4a40b1a4d55d6a62a3f8 in term 1.
I20260807 02:30:00.314400 8460 leader_election.cc:302] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8, 4a318aeac5534af280f8050cbf9f3eb6; no voters:
I20260807 02:30:00.314625 9089 raft_consensus.cc:2781] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:30:00.314854 9089 raft_consensus.cc:686] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 09eaae16e56d4a40b1a4d55d6a62a3f8, State: Running, Role: LEADER
I20260807 02:30:00.315177 9089 consensus_queue.cc:237] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:00.318673 8330 catalog_manager.cc:5495] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260807 02:30:00.330369 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.326162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.17.53.1:41617)
0807 02:30:00.326335 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:00.326341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:00.326357 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:30:00.326540 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:00.326543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:00.326602 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:00.326722 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:30:00.327008 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.327690 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.328809 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:00.329055 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:00.329173 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:00.329528 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:30:00.329932 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:30:00.330081 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:00.330151 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 02:30:00.330210 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:30:00.336213 8980 raft_consensus.cc:1261] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Refusing update from remote peer 09eaae16e56d4a40b1a4d55d6a62a3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:30:00.336232 8779 raft_consensus.cc:1261] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Refusing update from remote peer 09eaae16e56d4a40b1a4d55d6a62a3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:30:00.336970 9095 consensus_queue.cc:1035] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:30:00.337164 9089 consensus_queue.cc:1035] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:30:00.352653 8330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:30:00.352854 8330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:30:00.355422 8330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Sending DeleteTablet for 3 replicas of tablet 578414ce5f374aa3ae859d79618a3fc0
I20260807 02:30:00.356240 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 8430
I20260807 02:30:00.356148 8952 tablet_service.cc:1514] Processing DeleteTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:58992
I20260807 02:30:00.356228 8752 tablet_service.cc:1514] Processing DeleteTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:54340
I20260807 02:30:00.359768 9116 tablet_replica.cc:330] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: stopping tablet replica
I20260807 02:30:00.360116 9116 raft_consensus.cc:2229] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:30:00.360458 9116 raft_consensus.cc:2258] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:00.362932 9116 ts_tablet_manager.cc:1905] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:30:00.365566 9116 ts_tablet_manager.cc:1918] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:30:00.365710 9116 log.cc:1196] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/578414ce5f374aa3ae859d79618a3fc0
I20260807 02:30:00.366086 9115 tablet_replica.cc:330] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: stopping tablet replica
I20260807 02:30:00.366169 9116 ts_tablet_manager.cc:1939] T 578414ce5f374aa3ae859d79618a3fc0 P f61d76f543f24d0e83086f9f2996e3e7: Deleting consensus metadata
I20260807 02:30:00.366437 9115 raft_consensus.cc:2229] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:30:00.366771 9115 raft_consensus.cc:2258] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:00.367722 8309 catalog_manager.cc:4841] TS f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): tablet 578414ce5f374aa3ae859d79618a3fc0 (table test-table [id=93ffa7db973c4fc7a9097307ed63422b]) successfully deleted
I20260807 02:30:00.368369 9115 ts_tablet_manager.cc:1905] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:30:00.371838 9115 ts_tablet_manager.cc:1918] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:30:00.371950 9115 log.cc:1196] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/578414ce5f374aa3ae859d79618a3fc0
W20260807 02:30:00.372042 8306 connection.cc:535] client connection to 127.17.53.1:41617 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 02:30:00.372215 8306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 02:30:00.372294 9115 ts_tablet_manager.cc:1939] T 578414ce5f374aa3ae859d79618a3fc0 P 4a318aeac5534af280f8050cbf9f3eb6: Deleting consensus metadata
I20260807 02:30:00.372413 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 8648
I20260807 02:30:00.374876 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.373138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41617 (local address 127.0.0.1:49344)
0807 02:30:00.374708 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.374786 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1474}
W20260807 02:30:00.375042 8306 catalog_manager.cc:4584] TS 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 578414ce5f374aa3ae859d79618a3fc0: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41617: connect: Connection refused (error 111)
I20260807 02:30:00.385099 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 8855
I20260807 02:30:00.395462 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:00.385307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.0.0.1:54344)
0807 02:30:00.395216 (+ 9909us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:00.395313 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.2:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9756}
I20260807 02:30:00.405185 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 8274
I20260807 02:30:20.416249 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--webserver_interface=127.17.53.62
--webserver_port=33671
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:20.536259 9121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:20.536543 9121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:20.536585 9121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:20.540557 9121 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 02:30:20.540614 9121 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 02:30:20.540630 9121 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 02:30:20.540644 9121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:30:20.540663 9121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:20.540680 9121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:20.540696 9121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:30:20.540711 9121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:30:20.546042 9121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:20.547075 9121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:20.548111 9121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:30:20.563175 9121 server_base.cc:1034] running on GCE node
W20260807 02:30:20.563122 9132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:20.563141 9135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:20.563105 9133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:20.564172 9121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:20.565500 9121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:20.566674 9121 hybrid_clock.cc:648] HybridClock initialized: now 1786069820566537 us; error 150 us; skew 500 ppm
Aug 07 02:30:20 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:20.576464 9121 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:30:20.577705 9121 webserver.cc:460] Webserver started at http://127.17.53.62:33671/ using document root <none> and password file <none>
I20260807 02:30:20.578392 9121 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:20.578476 9121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:20.583063 9121 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:30:20.585503 9149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:20.586935 9121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:20.587103 9121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "f729d9b61fbb442aa25d2e3a0a177066"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "2d4d16d04e5d6341a73b8f238adc87fb"
server_key_iv: "97dd18fcda5105ba55508589a2f5977b"
server_key_version: "encryptionkey@0"
I20260807 02:30:20.587746 9121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:20.602905 9121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:20.606460 9121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:20.606706 9121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:20.614800 9121 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35283
I20260807 02:30:20.615154 9227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35283 every 8 connection(s)
I20260807 02:30:20.616075 9121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:30:20.621300 9229 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:30:20.623276 9229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap starting.
I20260807 02:30:20.625401 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 9121
I20260807 02:30:20.625869 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:41617
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=42215
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:30:20.642206 9229 log.cc:824] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:20.651533 9229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:30:20.652021 9229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap complete.
I20260807 02:30:20.656314 9229 raft_consensus.cc:348] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:20.656733 9229 raft_consensus.cc:729] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Initialized, Role: FOLLOWER
I20260807 02:30:20.657281 9229 consensus_queue.cc:260] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:20.657429 9229 raft_consensus.cc:388] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:30:20.657526 9229 raft_consensus.cc:482] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:30:20.657644 9229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 1 FOLLOWER]: Advancing to term 2
I20260807 02:30:20.659559 9229 raft_consensus.cc:504] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:20.659927 9229 leader_election.cc:302] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f729d9b61fbb442aa25d2e3a0a177066; no voters:
I20260807 02:30:20.660282 9229 leader_election.cc:288] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 02:30:20.660367 9235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 02:30:20.660629 9235 raft_consensus.cc:686] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 LEADER]: Becoming Leader. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Running, Role: LEADER
I20260807 02:30:20.661011 9235 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:20.661592 9229 sys_catalog.cc:564] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:30:20.662516 9236 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:30:20.662647 9236 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:20.663029 9237 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729d9b61fbb442aa25d2e3a0a177066. Latest consensus state: current_term: 2 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:30:20.663167 9237 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:20.663589 9247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:30:20.668500 9247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:30:20.669157 9247 catalog_manager.cc:1259] Loaded cluster ID: 261790c262ff46feaa5dfa8fb9d6da0d
I20260807 02:30:20.669231 9247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:30:20.672453 9247 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:30:20.673154 9247 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Loaded TSK: 0
I20260807 02:30:20.687808 9247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Generated new TSK 1
I20260807 02:30:20.688071 9247 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:30:20.752209 9233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:20.752452 9233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:20.752547 9233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:20.756595 9233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:20.756659 9233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:20.756714 9233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:30:20.762317 9233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:41617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=42215
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:20.763499 9233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:20.764642 9233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:20.775988 9268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:20.776000 9267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:20.776257 9233 server_base.cc:1034] running on GCE node
W20260807 02:30:20.776500 9270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:20.777004 9233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:20.777871 9233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:20.779106 9233 hybrid_clock.cc:648] HybridClock initialized: now 1786069820779081 us; error 34 us; skew 500 ppm
Aug 07 02:30:20 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:20.782788 9233 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:30:20.784034 9233 webserver.cc:460] Webserver started at http://127.17.53.1:42215/ using document root <none> and password file <none>
I20260807 02:30:20.784864 9233 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:20.784989 9233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:20.790148 9233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:30:20.792929 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:20.794600 9233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:20.794718 9233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "9ea24f3a43c44a364a99a92302d0e480"
server_key_iv: "3a42ee8799df6c2098dc9f31bf2e6709"
server_key_version: "encryptionkey@0"
I20260807 02:30:20.795419 9233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:20.812570 9233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:20.816358 9233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:20.816613 9233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:20.817425 9233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:20.818791 9294 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 02:30:20.823280 9233 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 02:30:20.823374 9233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260807 02:30:20.823434 9233 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 02:30:20.825217 9233 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 02:30:20.825266 9233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 02:30:20.825619 9294 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap starting.
I20260807 02:30:20.837548 9233 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:41617
I20260807 02:30:20.837671 9468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:41617 every 8 connection(s)
I20260807 02:30:20.838852 9233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:30:20.844247 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 9233
I20260807 02:30:20.844795 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:39377
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=37033
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:30:20.847757 9294 log.cc:824] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:30:20 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:30:20.856906 9294 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:30:20.857372 9294 tablet_bootstrap.cc:492] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap complete.
I20260807 02:30:20.859170 9294 ts_tablet_manager.cc:1397] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.009s
I20260807 02:30:20.861181 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:20.841490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:39931 (local address 127.17.53.62:35283)
0807 02:30:20.841940 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:20.841962 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:20.842423 (+ 461us) server_negotiation.cc:407] Connection header received
0807 02:30:20.843987 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:20.843999 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:20.844376 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:20.844771 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:20.846247 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.847705 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.849454 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.850262 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.850473 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:20.853883 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:20.853893 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:20.853917 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:20.853963 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:20.856455 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:20.857264 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:20.857273 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:20.857281 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:20.857473 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:20.857929 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:20.857935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:20.857940 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:20.858572 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:20.858736 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:20.859718 (+ 982us) server_negotiation.cc:299] Negotiation successful
0807 02:30:20.860123 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":145,"threads_started":1}
I20260807 02:30:20.862936 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:20.841435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:39931)
0807 02:30:20.842220 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:20.842285 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:20.843512 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:20.845133 (+ 1621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:20.845149 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:20.845528 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:20.846039 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:20.846055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.847910 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.847913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:20.849285 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:20.849305 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.850473 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.850481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:20.850753 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:20.851532 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:20.851561 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:20.853600 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:20.856631 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:20.856637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:20.856651 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:20.857088 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:20.857591 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:20.857595 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:20.857597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:20.857792 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:20.858819 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:20.858829 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:20.859259 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 02:30:20.861593 (+ 2334us) client_negotiation.cc:240] Negotiation successful
0807 02:30:20.861954 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":164,"threads_started":1}
I20260807 02:30:20.862649 9294 raft_consensus.cc:348] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:20.863350 9294 raft_consensus.cc:729] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09eaae16e56d4a40b1a4d55d6a62a3f8, State: Initialized, Role: FOLLOWER
I20260807 02:30:20.863873 9294 consensus_queue.cc:260] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } } peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } }
I20260807 02:30:20.865072 9294 ts_tablet_manager.cc:1428] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 02:30:20.865978 9471 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:30:20.866279 9471 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:20.866817 9471 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
I20260807 02:30:20.869657 9177 ts_manager.cc:194] Registered new tserver with Master: 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617)
I20260807 02:30:20.874032 9177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:39931
I20260807 02:30:20.877497 9471 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
Aug 07 02:30:20 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.1@KRBTEST.COM
I20260807 02:30:20.884552 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:20.873383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41617 (local address 127.0.0.1:48202)
0807 02:30:20.873828 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:20.873878 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:20.874018 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:20.874825 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:20.874831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:20.875347 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:20.875467 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:20.875474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.876477 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.876481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:20.877235 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:20.877250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.878050 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.878054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:20.878183 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:20.878895 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:20.878918 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:20.880438 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:20.882461 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:20.882471 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:20.882478 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:20.882734 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:20.883070 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:20.883073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:20.883075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:20.883169 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:20.883660 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:20.883710 (+ 50us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:20.883922 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 02:30:20.884105 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 02:30:20.884356 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":141,"threads_started":1}
I20260807 02:30:20.884613 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:20.873452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.17.53.1:41617)
0807 02:30:20.874310 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:20.874325 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:20.874346 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:30:20.874420 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:20.874424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:20.874604 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:20.874823 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:20.875602 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.876347 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.877406 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:20.877892 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:20.878017 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:20.880592 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:20.880609 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:20.880620 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:20.880644 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:20.882330 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:20.882865 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:20.882876 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:20.882877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:20.882960 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:20.883265 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:20.883268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:20.883270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:20.883535 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:20.883688 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:20.884264 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 02:30:20.884462 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":105,"threads_started":1}
I20260807 02:30:20.885980 9363 tablet_service.cc:1514] Processing DeleteTablet for tablet 578414ce5f374aa3ae859d79618a3fc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48202
I20260807 02:30:20.887189 9487 tablet_replica.cc:330] stopping tablet replica
I20260807 02:30:20.887622 9487 raft_consensus.cc:2229] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:30:20.887959 9487 raft_consensus.cc:2258] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:20.889308 9487 ts_tablet_manager.cc:1905] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:30:20.895049 9487 ts_tablet_manager.cc:1918] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:30:20.895141 9487 log.cc:1196] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal/wals/578414ce5f374aa3ae859d79618a3fc0
I20260807 02:30:20.895372 9487 ts_tablet_manager.cc:1939] T 578414ce5f374aa3ae859d79618a3fc0 P 09eaae16e56d4a40b1a4d55d6a62a3f8: Deleting consensus metadata
I20260807 02:30:20.897572 9152 catalog_manager.cc:4841] TS 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617): tablet 578414ce5f374aa3ae859d79618a3fc0 (table test-table [id=93ffa7db973c4fc7a9097307ed63422b]) successfully deleted
W20260807 02:30:20.983372 9477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:20.983618 9477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:20.983660 9477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:20.987715 9477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:20.987773 9477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:20.987862 9477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:30:20.993352 9477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:39377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=37033
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:20.994405 9477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:20.995452 9477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:21.005787 9498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:21.005800 9500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:21.005791 9495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:21.006225 9477 server_base.cc:1034] running on GCE node
I20260807 02:30:21.006699 9477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:21.007383 9477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:21.008561 9477 hybrid_clock.cc:648] HybridClock initialized: now 1786069821008541 us; error 29 us; skew 500 ppm
Aug 07 02:30:21 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069821, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:21.011498 9477 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:30:21.012449 9477 webserver.cc:460] Webserver started at http://127.17.53.2:37033/ using document root <none> and password file <none>
I20260807 02:30:21.013120 9477 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:21.013236 9477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:21.017175 9477 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 02:30:21.019333 9507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.020542 9477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:30:21.020689 9477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "4a318aeac5534af280f8050cbf9f3eb6"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "df628ad028d360599e35b2bd2bd59f4a"
server_key_iv: "a87934c494f43416a60f2a4b534b7191"
server_key_version: "encryptionkey@0"
I20260807 02:30:21.021344 9477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:21.060149 9477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:21.063895 9477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:21.064103 9477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:21.064949 9477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:21.065953 9477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:21.066027 9477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.066097 9477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:21.066128 9477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.075456 9477 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39377
I20260807 02:30:21.075479 9672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39377 every 8 connection(s)
I20260807 02:30:21.076484 9477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:30:21.080132 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 9477
I20260807 02:30:21.080560 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:45791
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=40101
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 02:30:21 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069821, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:30:21.093166 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.078355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:44399 (local address 127.17.53.62:35283)
0807 02:30:21.078512 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.078516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.079348 (+ 832us) server_negotiation.cc:407] Connection header received
0807 02:30:21.080739 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.080743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.080861 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.080970 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:21.082141 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.082680 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.083906 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.084153 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.084264 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.087673 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:21.087677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:21.087679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:21.087710 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:21.089919 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.090666 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.090672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.090675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.090788 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.091341 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.091348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.091352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.091675 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:21.091794 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.092756 (+ 962us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.092952 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:30:21.093991 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.078700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:44399)
0807 02:30:21.079159 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.079201 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.080515 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.081152 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.081158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.081516 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:21.081984 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.081998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.082836 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.082847 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.083766 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.083781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.084328 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.084335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.084571 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.085387 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:21.085415 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:21.087439 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:21.090080 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:21.090089 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:21.090106 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:21.090527 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:21.090915 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:21.090919 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:21.090921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:21.091147 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:21.091932 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:21.091940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:21.092299 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.092699 (+ 400us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.093098 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260807 02:30:21.095399 9674 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:30:21.095659 9674 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:21.096248 9674 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
I20260807 02:30:21.097509 9177 ts_manager.cc:194] Registered new tserver with Master: 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
I20260807 02:30:21.098249 9177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:44399
W20260807 02:30:21.199095 9677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:21.199342 9677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:21.199383 9677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:21.203632 9677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:21.203691 9677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:21.203749 9677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:30:21.208841 9677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:45791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=40101
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:21.209789 9677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:21.210726 9677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:21.220782 9683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:21.220808 9682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:21.220956 9677 server_base.cc:1034] running on GCE node
W20260807 02:30:21.220745 9686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:21.221542 9677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:21.222184 9677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:21.223378 9677 hybrid_clock.cc:648] HybridClock initialized: now 1786069821223343 us; error 46 us; skew 500 ppm
Aug 07 02:30:21 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069821, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:21.226408 9677 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:30:21.227308 9677 webserver.cc:460] Webserver started at http://127.17.53.3:40101/ using document root <none> and password file <none>
I20260807 02:30:21.227896 9677 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:21.228012 9677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:21.232051 9677 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:30:21.234500 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.235942 9677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:30:21.236089 9677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "f61d76f543f24d0e83086f9f2996e3e7"
format_stamp: "Formatted at 2026-08-07 02:30:00 on dist-test-slave-g170"
server_key: "edf3536ac13d2fcc44aef4b16da9aa5e"
server_key_iv: "7f3f19b3556632aec8c6df69d9d0d74c"
server_key_version: "encryptionkey@0"
I20260807 02:30:21.236641 9677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:21.265321 9677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:21.269371 9677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:21.269605 9677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:21.270278 9677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:21.271224 9677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:21.271275 9677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.271346 9677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:21.271386 9677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:21.281245 9677 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:45791
I20260807 02:30:21.281322 9873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:45791 every 8 connection(s)
I20260807 02:30:21.282383 9677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:30:21.289080 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 9677
Aug 07 02:30:21 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069821, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:30:21.298735 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.284386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:57175 (local address 127.17.53.62:35283)
0807 02:30:21.284540 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.284544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.285504 (+ 960us) server_negotiation.cc:407] Connection header received
0807 02:30:21.287000 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.287004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.287065 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.287189 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:21.288448 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.289304 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.290973 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.291314 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.291404 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.294456 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:21.294461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:21.294466 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:21.294494 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:21.296187 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.296858 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.296861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.296862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.296938 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.297374 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.297377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.297379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.297611 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:21.297779 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.298434 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.298566 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 02:30:21.298971 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.289858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.17.53.62:35283)
0807 02:30:21.290128 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.290133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.290328 (+ 195us) server_negotiation.cc:407] Connection header received
0807 02:30:21.290458 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.290460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.290500 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.290581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:21.291606 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.292296 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.293497 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.293861 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.293943 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.295329 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:21.295333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:21.295335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:21.295380 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:21.297133 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.297612 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.297615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.297616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.297735 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.298039 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.298044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.298045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.298243 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:21.298468 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.298658 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.298784 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260807 02:30:21.299885 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.284818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:57175)
0807 02:30:21.285343 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.285390 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.286772 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.287394 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.287401 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.287779 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:21.288291 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.288306 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.289465 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.289468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.290819 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.290833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.291484 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.291489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.291710 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.292331 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:21.292348 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:21.294266 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:21.296331 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:21.296339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:21.296354 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:21.296712 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:21.297038 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:21.297043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:21.297046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:21.297252 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:21.297814 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:21.297822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:21.298121 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.298343 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.298716 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":126,"threads_started":1}
I20260807 02:30:21.301403 9874 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35283
I20260807 02:30:21.301625 9874 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:21.302235 9874 heartbeater.cc:511] Master 127.17.53.62:35283 requested a full tablet report, sending...
I20260807 02:30:21.303254 9177 ts_manager.cc:194] Registered new tserver with Master: f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791)
I20260807 02:30:21.303795 9177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:57175
I20260807 02:30:21.313087 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:30:21.317015 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.313708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.17.53.62:35283)
0807 02:30:21.313867 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.313872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.314080 (+ 208us) server_negotiation.cc:407] Connection header received
0807 02:30:21.314220 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.314222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.314313 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.314400 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:30:21.314601 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.315099 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.315950 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.316177 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.316254 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.316513 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:30:21.316635 (+ 122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 02:30:21.316897 (+ 262us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47532: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 02:30:21.317090 9880 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47532: token expired
W20260807 02:30:21.317425 9056 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.53.62:35283)
I20260807 02:30:21.324875 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.317776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.17.53.62:35283)
0807 02:30:21.317882 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.317892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.317906 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:30:21.317936 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.317938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.317969 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.318032 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:21.318657 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.319120 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.319895 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.320096 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.320181 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.321592 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:21.321595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:21.321597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:21.321620 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:21.323240 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.323677 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.323680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.323681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.323771 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:21.324032 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:21.324034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:21.324035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:21.324200 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:21.324301 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.324544 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.324649 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 02:30:21.327422 9177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:30:21.330852 9177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:30:21.339313 9363 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea3667c0a84522b723b13a8175cf3b (DEFAULT_TABLE table=test-table [id=55a0545d01ca4461bcd4399c314a033c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:21.340049 9363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea3667c0a84522b723b13a8175cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:21.343161 9294 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap starting.
I20260807 02:30:21.343822 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.338255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.0.0.1:36668)
0807 02:30:21.338797 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.338811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.338900 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.339564 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.339567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.339585 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:21.339668 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.339687 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.341066 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.341073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.342153 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.342182 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.343352 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.343356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.343455 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.343503 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.343590 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.343681 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":116,"threads_started":1}
I20260807 02:30:21.344049 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.337858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:45791 (local address 127.0.0.1:38986)
0807 02:30:21.337980 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.337992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.338082 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.339226 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.339229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.339249 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:21.339363 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.339370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.340870 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.340874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.342229 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.342240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.343642 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.343646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.343741 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.343764 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.343832 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.343923 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":162}
I20260807 02:30:21.344185 9294 tablet_bootstrap.cc:654] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:21.344399 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.338449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.17.53.2:39377)
0807 02:30:21.338946 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.338957 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.338977 (+ 20us) server_negotiation.cc:407] Connection header received
0807 02:30:21.339048 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.339052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.339227 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.339378 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:21.339828 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.340957 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.342308 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.342983 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.343119 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.343281 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.344119 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.344230 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":160,"threads_started":1}
I20260807 02:30:21.344427 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.338285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.17.53.3:45791)
0807 02:30:21.338703 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.338717 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.338742 (+ 25us) server_negotiation.cc:407] Connection header received
0807 02:30:21.338830 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.338836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.339059 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.339237 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:21.339472 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.340731 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.342473 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.343499 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.343643 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.343795 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.343858 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.344299 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":186,"threads_started":1}
I20260807 02:30:21.345983 9294 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: No bootstrap required, opened a new log
I20260807 02:30:21.346086 9294 ts_tablet_manager.cc:1397] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:21.346573 9294 raft_consensus.cc:348] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.346720 9294 raft_consensus.cc:374] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:21.346774 9294 raft_consensus.cc:729] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09eaae16e56d4a40b1a4d55d6a62a3f8, State: Initialized, Role: FOLLOWER
I20260807 02:30:21.346952 9294 consensus_queue.cc:260] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.347256 9294 ts_tablet_manager.cc:1428] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 02:30:21.347935 9777 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea3667c0a84522b723b13a8175cf3b (DEFAULT_TABLE table=test-table [id=55a0545d01ca4461bcd4399c314a033c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:21.347949 9573 tablet_service.cc:1467] Processing CreateTablet for tablet 2eea3667c0a84522b723b13a8175cf3b (DEFAULT_TABLE table=test-table [id=55a0545d01ca4461bcd4399c314a033c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:30:21.349160 9777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea3667c0a84522b723b13a8175cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:21.349222 9573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eea3667c0a84522b723b13a8175cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:21.356711 9903 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Bootstrap starting.
I20260807 02:30:21.356817 9906 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Bootstrap starting.
I20260807 02:30:21.359326 9903 tablet_bootstrap.cc:654] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:21.359951 9906 tablet_bootstrap.cc:654] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:21.360553 9903 log.cc:824] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:21.360837 9906 log.cc:824] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:21.362911 9906 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: No bootstrap required, opened a new log
I20260807 02:30:21.362923 9903 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: No bootstrap required, opened a new log
I20260807 02:30:21.363178 9906 ts_tablet_manager.cc:1397] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 02:30:21.363198 9903 ts_tablet_manager.cc:1397] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 02:30:21.366572 9903 raft_consensus.cc:348] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.366894 9903 raft_consensus.cc:374] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:21.366696 9906 raft_consensus.cc:348] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.366969 9903 raft_consensus.cc:729] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f61d76f543f24d0e83086f9f2996e3e7, State: Initialized, Role: FOLLOWER
I20260807 02:30:21.366982 9906 raft_consensus.cc:374] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:21.367081 9906 raft_consensus.cc:729] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a318aeac5534af280f8050cbf9f3eb6, State: Initialized, Role: FOLLOWER
I20260807 02:30:21.367455 9903 consensus_queue.cc:260] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.367540 9906 consensus_queue.cc:260] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.368198 9874 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
I20260807 02:30:21.368304 9674 heartbeater.cc:503] Master 127.17.53.62:35283 was elected leader, sending a full tablet report...
I20260807 02:30:21.368589 9903 ts_tablet_manager.cc:1428] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 02:30:21.369465 9906 ts_tablet_manager.cc:1428] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260807 02:30:21.533468 9877 tablet.cc:2369] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:30:21.577749 9675 tablet.cc:2369] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:30:21.590344 9472 tablet.cc:2369] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:30:21.676160 9912 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:21.676481 9912 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.677841 9912 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377), 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617)
I20260807 02:30:21.682000 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.678155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.17.53.3:46967)
0807 02:30:21.678332 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.678347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.678510 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.678851 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.678854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.678875 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:21.679019 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.679026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.679841 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.679847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.680731 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.680741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.681515 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.681520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.681637 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.681668 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.681773 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.681851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 02:30:21.682000 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.678315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:46967 (local address 127.17.53.2:39377)
0807 02:30:21.678489 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.678495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.678514 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:30:21.678658 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.678662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.678733 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.678835 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:21.679151 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.679714 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.680922 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.681383 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.681510 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.681581 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.681794 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.681868 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 02:30:21.682703 9601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea3667c0a84522b723b13a8175cf3b" candidate_uuid: "f61d76f543f24d0e83086f9f2996e3e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a318aeac5534af280f8050cbf9f3eb6" is_pre_election: true
I20260807 02:30:21.682790 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.678583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41617 (local address 127.17.53.3:32997)
0807 02:30:21.678998 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.679015 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:21.679124 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:21.679486 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:21.679490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:21.679507 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:30:21.679640 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.679646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.680461 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.680466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.681337 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:21.681346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.682319 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.682323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:21.682442 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.682463 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 02:30:21.682572 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 02:30:21.682644 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":172,"threads_started":1}
I20260807 02:30:21.682886 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.678686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:32997 (local address 127.17.53.1:41617)
0807 02:30:21.679066 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.679073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.679090 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:30:21.679272 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.679275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.679348 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.679477 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:30:21.679779 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.680333 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.681488 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.681998 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.682148 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.682346 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.682595 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.682731 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":87,"threads_started":1}
I20260807 02:30:21.683069 9601 raft_consensus.cc:2454] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f61d76f543f24d0e83086f9f2996e3e7 in term 0.
I20260807 02:30:21.683681 9701 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7; no voters:
I20260807 02:30:21.683625 9391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea3667c0a84522b723b13a8175cf3b" candidate_uuid: "f61d76f543f24d0e83086f9f2996e3e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" is_pre_election: true
I20260807 02:30:21.683955 9391 raft_consensus.cc:2454] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f61d76f543f24d0e83086f9f2996e3e7 in term 0.
I20260807 02:30:21.684000 9912 raft_consensus.cc:2781] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:30:21.684095 9912 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:30:21.684183 9912 raft_consensus.cc:3037] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:21.685351 9912 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.685719 9912 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [CANDIDATE]: Term 1 election: Requested vote from peers 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377), 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617)
I20260807 02:30:21.686122 9601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea3667c0a84522b723b13a8175cf3b" candidate_uuid: "f61d76f543f24d0e83086f9f2996e3e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a318aeac5534af280f8050cbf9f3eb6"
I20260807 02:30:21.686151 9391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eea3667c0a84522b723b13a8175cf3b" candidate_uuid: "f61d76f543f24d0e83086f9f2996e3e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
I20260807 02:30:21.686254 9601 raft_consensus.cc:3037] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:21.686292 9391 raft_consensus.cc:3037] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:21.687287 9601 raft_consensus.cc:2454] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f61d76f543f24d0e83086f9f2996e3e7 in term 1.
I20260807 02:30:21.687541 9391 raft_consensus.cc:2454] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f61d76f543f24d0e83086f9f2996e3e7 in term 1.
I20260807 02:30:21.687577 9701 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7; no voters:
I20260807 02:30:21.687773 9912 raft_consensus.cc:2781] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:30:21.688031 9912 raft_consensus.cc:686] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 1 LEADER]: Becoming Leader. State: Replica: f61d76f543f24d0e83086f9f2996e3e7, State: Running, Role: LEADER
I20260807 02:30:21.688585 9912 consensus_queue.cc:237] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:21.691907 9175 catalog_manager.cc:5495] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "f61d76f543f24d0e83086f9f2996e3e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } health_report { overall_health: UNKNOWN } } }
I20260807 02:30:21.713797 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.710079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.17.53.3:45791)
0807 02:30:21.710245 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:21.710249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:21.710267 (+ 18us) server_negotiation.cc:407] Connection header received
0807 02:30:21.710367 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:21.710370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:21.710427 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:21.710527 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:30:21.710842 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.711469 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.712338 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:21.712550 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:21.712651 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:21.712995 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:30:21.713434 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:30:21.713530 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:21.713607 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 02:30:21.713657 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 02:30:21.719998 9601 raft_consensus.cc:1261] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Refusing update from remote peer f61d76f543f24d0e83086f9f2996e3e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:30:21.720016 9391 raft_consensus.cc:1261] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Refusing update from remote peer f61d76f543f24d0e83086f9f2996e3e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:30:21.720706 9912 consensus_queue.cc:1035] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:30:21.721058 9916 consensus_queue.cc:1035] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:30:21.728756 9923 mvcc.cc:205] Tried to move back new op lower bound from 7315741989756981248 to 7315741989639122944. Current Snapshot: MvccSnapshot[applied={T|T < 7315741989756981248}]
I20260807 02:30:21.731271 9924 mvcc.cc:205] Tried to move back new op lower bound from 7315741989756981248 to 7315741989639122944. Current Snapshot: MvccSnapshot[applied={T|T < 7315741989756981248 or (T in {7315741989756981248})}]
I20260807 02:30:21.737699 9173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:30:21.737854 9173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:30:21.740361 9173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Sending DeleteTablet for 3 replicas of tablet 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:21.741046 9777 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eea3667c0a84522b723b13a8175cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:21 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:38986
I20260807 02:30:21.741411 9363 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eea3667c0a84522b723b13a8175cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:21 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:48202
I20260807 02:30:21.741483 9573 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eea3667c0a84522b723b13a8175cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:30:21 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:36668
I20260807 02:30:21.741580 9940 tablet_replica.cc:330] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: stopping tablet replica
I20260807 02:30:21.741824 9942 tablet_replica.cc:330] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: stopping tablet replica
I20260807 02:30:21.741906 9940 raft_consensus.cc:2229] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:30:21.741942 9942 raft_consensus.cc:2229] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:30:21.742074 9942 raft_consensus.cc:2258] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:21.742180 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 9233
I20260807 02:30:21.742334 9940 raft_consensus.cc:2258] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:21.750445 9943 tablet_replica.cc:330] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: stopping tablet replica
I20260807 02:30:21.750836 9943 raft_consensus.cc:2229] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:30:21.751221 9943 raft_consensus.cc:2258] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:30:21.751387 9940 ts_tablet_manager.cc:1905] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:30:21.752728 9943 ts_tablet_manager.cc:1905] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:30:21.755627 9943 ts_tablet_manager.cc:1918] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:30:21.755744 9943 log.cc:1196] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:21.755949 9940 ts_tablet_manager.cc:1918] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:30:21.756116 9940 log.cc:1196] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:21.756134 9943 ts_tablet_manager.cc:1939] T 2eea3667c0a84522b723b13a8175cf3b P 4a318aeac5534af280f8050cbf9f3eb6: Deleting consensus metadata
I20260807 02:30:21.756603 9940 ts_tablet_manager.cc:1939] T 2eea3667c0a84522b723b13a8175cf3b P f61d76f543f24d0e83086f9f2996e3e7: Deleting consensus metadata
I20260807 02:30:21.757615 9152 catalog_manager.cc:4841] TS 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): tablet 2eea3667c0a84522b723b13a8175cf3b (table test-table [id=55a0545d01ca4461bcd4399c314a033c]) successfully deleted
I20260807 02:30:21.757979 9153 catalog_manager.cc:4841] TS f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): tablet 2eea3667c0a84522b723b13a8175cf3b (table test-table [id=55a0545d01ca4461bcd4399c314a033c]) successfully deleted
W20260807 02:30:21.758474 9152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:41617 (error 108)
I20260807 02:30:21.758773 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 9477
I20260807 02:30:21.770187 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:21.769845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:41617 (local address 127.0.0.1:48216)
0807 02:30:21.770005 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:21.770101 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 02:30:21.770447 9152 catalog_manager.cc:4584] TS 09eaae16e56d4a40b1a4d55d6a62a3f8 (127.17.53.1:41617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2eea3667c0a84522b723b13a8175cf3b: Network error: Client connection negotiation failed: client connection to 127.17.53.1:41617: connect: Connection refused (error 111)
I20260807 02:30:21.777395 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 9677
I20260807 02:30:21.787075 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 9121
I20260807 02:30:41.798918 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--webserver_interface=127.17.53.62
--webserver_port=33671
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:41.919687 9946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:41.919977 9946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:41.920019 9946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:41.924278 9946 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 02:30:41.924348 9946 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 02:30:41.924365 9946 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 02:30:41.924381 9946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:30:41.924399 9946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:41.924417 9946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:41.924433 9946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:30:41.924448 9946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:30:41.929965 9946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=33671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:41.931075 9946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:41.932159 9946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:41.941773 9954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:41.941773 9957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:41.941777 9953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:41.942322 9946 server_base.cc:1034] running on GCE node
I20260807 02:30:41.942898 9946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:41.944226 9946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:41.945417 9946 hybrid_clock.cc:648] HybridClock initialized: now 1786069841945388 us; error 46 us; skew 500 ppm
Aug 07 02:30:41 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:41.953709 9946 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:30:41.954807 9946 webserver.cc:460] Webserver started at http://127.17.53.62:33671/ using document root <none> and password file <none>
I20260807 02:30:41.955462 9946 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:41.955566 9946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:41.959851 9946 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:30:41.962597 9964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:41.964064 9946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:30:41.964223 9946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "f729d9b61fbb442aa25d2e3a0a177066"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "2d4d16d04e5d6341a73b8f238adc87fb"
server_key_iv: "97dd18fcda5105ba55508589a2f5977b"
server_key_version: "encryptionkey@0"
I20260807 02:30:41.964861 9946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:41.975456 9946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:41.978912 9946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:41.979121 9946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:41.987185 9946 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35283
I20260807 02:30:41.987540 10050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35283 every 8 connection(s)
I20260807 02:30:41.988369 9946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:30:41.993445 10051 sys_catalog.cc:263] Verifying existing consensus state
I20260807 02:30:41.995369 10051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap starting.
I20260807 02:30:41.998241 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 9946
I20260807 02:30:41.998790 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:41617
--local_ip_for_outbound_sockets=127.17.53.1
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=42215
--webserver_interface=127.17.53.1
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:30:42.017305 10051 log.cc:824] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:42.030870 10051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:30:42.031402 10051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Bootstrap complete.
I20260807 02:30:42.036358 10051 raft_consensus.cc:348] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:42.036957 10051 raft_consensus.cc:729] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Initialized, Role: FOLLOWER
I20260807 02:30:42.037609 10051 consensus_queue.cc:260] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:42.037793 10051 raft_consensus.cc:388] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:30:42.037896 10051 raft_consensus.cc:482] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:30:42.038038 10051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 2 FOLLOWER]: Advancing to term 3
I20260807 02:30:42.040329 10051 raft_consensus.cc:504] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:42.040742 10051 leader_election.cc:302] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f729d9b61fbb442aa25d2e3a0a177066; no voters:
I20260807 02:30:42.041260 10051 leader_election.cc:288] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 02:30:42.041365 10057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 02:30:42.041607 10057 raft_consensus.cc:686] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [term 3 LEADER]: Becoming Leader. State: Replica: f729d9b61fbb442aa25d2e3a0a177066, State: Running, Role: LEADER
I20260807 02:30:42.041958 10057 consensus_queue.cc:237] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } }
I20260807 02:30:42.042760 10051 sys_catalog.cc:564] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:30:42.043497 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:30:42.043630 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:42.044349 10060 sys_catalog.cc:455] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f729d9b61fbb442aa25d2e3a0a177066. Latest consensus state: current_term: 3 leader_uuid: "f729d9b61fbb442aa25d2e3a0a177066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f729d9b61fbb442aa25d2e3a0a177066" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35283 } } }
I20260807 02:30:42.044454 10060 sys_catalog.cc:458] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:42.044792 10064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:30:42.050990 10064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:30:42.051918 10064 catalog_manager.cc:1259] Loaded cluster ID: 261790c262ff46feaa5dfa8fb9d6da0d
I20260807 02:30:42.052017 10064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:30:42.055119 10064 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:30:42.056032 10064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Loaded TSK: 0
I20260807 02:30:42.056137 10064 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f729d9b61fbb442aa25d2e3a0a177066: Loaded TSK: 1
F20260807 02:30:42.057199 10064 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8e887779ad google::LogMessage::Fail() at ??:0
@ 0x7f8e8877a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8e887774de google::LogMessage::Flush() at ??:0
@ 0x7f8e88778a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8e8b43b18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8e8b433f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8e930dc7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8e930b699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8e930b7178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f8e930f8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8e9311782b std::function<>::operator()() at ??:0
@ 0x7f8e930b6b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8e930b7f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8e930b4579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8e930f8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8e8a9c93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8e8a9c9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8e8a9cb3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8e8a9ba512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8e8bffe6db start_thread at ??:0
@ 0x7f8e8727561f clone at ??:0
W20260807 02:30:42.165537 10053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:42.165853 10053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:42.165940 10053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:42.170382 10053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:42.170452 10053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:42.170574 10053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:30:42.177678 10053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:41617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=42215
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:42.178987 10053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:42.180145 10053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:42.190928 10125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:42.190932 10121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:42.191149 10053 server_base.cc:1034] running on GCE node
W20260807 02:30:42.190932 10120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:42.191804 10053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:42.192618 10053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:42.193812 10053 hybrid_clock.cc:648] HybridClock initialized: now 1786069842193799 us; error 31 us; skew 500 ppm
Aug 07 02:30:42 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069842, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:42.197023 10053 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:30:42.198037 10053 webserver.cc:460] Webserver started at http://127.17.53.1:42215/ using document root <none> and password file <none>
I20260807 02:30:42.198681 10053 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:42.198782 10053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:42.203050 10053 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 02:30:42.205514 10134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.206938 10053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:30:42.207083 10053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "9ea24f3a43c44a364a99a92302d0e480"
server_key_iv: "3a42ee8799df6c2098dc9f31bf2e6709"
server_key_version: "encryptionkey@0"
I20260807 02:30:42.207662 10053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:42.228410 10053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:42.232718 10053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:42.233026 10053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:42.233721 10053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:42.234961 10144 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 02:30:42.241101 10053 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 02:30:42.241210 10053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260807 02:30:42.241322 10053 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 02:30:42.243122 10053 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 02:30:42.243196 10053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 02:30:42.243532 10144 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap starting.
I20260807 02:30:42.255122 10053 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:41617
I20260807 02:30:42.255295 10312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:41617 every 8 connection(s)
I20260807 02:30:42.256338 10053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:30:42.258442 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 10053
I20260807 02:30:42.258996 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:39377
--local_ip_for_outbound_sockets=127.17.53.2
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=37033
--webserver_interface=127.17.53.2
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:30:42.267760 10144 log.cc:824] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:42.279129 10144 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 02:30:42.279587 10144 tablet_bootstrap.cc:492] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Bootstrap complete.
I20260807 02:30:42.281342 10144 ts_tablet_manager.cc:1397] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent bootstrapping tablet: real 0.038s user 0.016s sys 0.020s
I20260807 02:30:42.284798 10144 raft_consensus.cc:348] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:42.285424 10144 raft_consensus.cc:729] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09eaae16e56d4a40b1a4d55d6a62a3f8, State: Initialized, Role: FOLLOWER
I20260807 02:30:42.286006 10144 consensus_queue.cc:260] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:42.287246 10144 ts_tablet_manager.cc:1428] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260807 02:30:42.319011 10318 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:30:42.258661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:33405)
0807 02:30:42.260242 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.260305 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:42.261558 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:42.318026 (+ 56468us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":184,"threads_started":1}
W20260807 02:30:42.319803 10316 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:35283 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:35283: Client connection negotiation failed: client connection to 127.17.53.62:35283: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 02:30:42.320727 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.320394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:58503)
0807 02:30:42.320550 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.320646 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 02:30:42.321592 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.321333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:44265)
0807 02:30:42.321470 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.321527 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 02:30:42.321866 10316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:30:42.391443 10320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:42.391696 10320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:42.391741 10320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:42.395735 10320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:42.395802 10320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:42.395903 10320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:30:42.401508 10320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:39377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=37033
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:42.402751 10320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:42.403962 10320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:42.414964 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:42.414997 10332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:42.415010 10333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:42.415223 10320 server_base.cc:1034] running on GCE node
I20260807 02:30:42.415789 10320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:42.416549 10320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:42.417724 10320 hybrid_clock.cc:648] HybridClock initialized: now 1786069842417703 us; error 35 us; skew 500 ppm
Aug 07 02:30:42 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069842, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:42.420977 10320 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:30:42.422060 10320 webserver.cc:460] Webserver started at http://127.17.53.2:37033/ using document root <none> and password file <none>
I20260807 02:30:42.422747 10320 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:42.422817 10320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:42.427309 10320 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:30:42.429737 10347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.431200 10320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:42.431356 10320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "4a318aeac5534af280f8050cbf9f3eb6"
format_stamp: "Formatted at 2026-08-07 02:29:59 on dist-test-slave-g170"
server_key: "df628ad028d360599e35b2bd2bd59f4a"
server_key_iv: "a87934c494f43416a60f2a4b534b7191"
server_key_version: "encryptionkey@0"
I20260807 02:30:42.431960 10320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:42.456426 10320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:42.460692 10320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:42.460985 10320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:42.461742 10320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:42.462785 10320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:42.462867 10320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.462951 10320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:42.463001 10320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.474114 10320 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39377
I20260807 02:30:42.474148 10516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39377 every 8 connection(s)
I20260807 02:30:42.475263 10320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:30:42.478458 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 10320
I20260807 02:30:42.478991 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:45791
--local_ip_for_outbound_sockets=127.17.53.3
--tserver_master_addrs=127.17.53.62:35283
--webserver_port=40101
--webserver_interface=127.17.53.3
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:30:42.480050 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.477842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:36025)
0807 02:30:42.478779 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.478947 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":120,"threads_started":1}
W20260807 02:30:42.480587 10518 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:35283 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:35283: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
I20260807 02:30:42.481375 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.481119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:40093)
0807 02:30:42.481249 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.481316 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 02:30:42.482174 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.481852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:56421)
0807 02:30:42.482007 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.482088 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 02:30:42.482404 10518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 02:30:42.602108 10525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:42.602443 10525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:42.602526 10525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:42.606658 10525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:42.606751 10525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:42.606845 10525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:30:42.612555 10525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:45791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=40101
--tserver_master_addrs=127.17.53.62:35283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:42.613763 10525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:42.614885 10525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:42.625162 10537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:42.625123 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:42.625134 10534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:42.625409 10525 server_base.cc:1034] running on GCE node
I20260807 02:30:42.625876 10525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:42.626513 10525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:42.627679 10525 hybrid_clock.cc:648] HybridClock initialized: now 1786069842627664 us; error 28 us; skew 500 ppm
Aug 07 02:30:42 dist-test-slave-g170 krb5kdc[8254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069842, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:42.630823 10525 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:30:42.631677 10525 webserver.cc:460] Webserver started at http://127.17.53.3:40101/ using document root <none> and password file <none>
I20260807 02:30:42.632287 10525 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:42.632354 10525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:42.636693 10525 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:30:42.639179 10551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.640985 10525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 02:30:42.641117 10525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "f61d76f543f24d0e83086f9f2996e3e7"
format_stamp: "Formatted at 2026-08-07 02:30:00 on dist-test-slave-g170"
server_key: "edf3536ac13d2fcc44aef4b16da9aa5e"
server_key_iv: "7f3f19b3556632aec8c6df69d9d0d74c"
server_key_version: "encryptionkey@0"
I20260807 02:30:42.641743 10525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:42.657588 10525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:42.661139 10525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:42.661307 10525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:42.661918 10525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:42.662771 10525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:42.662827 10525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.662878 10525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:42.662894 10525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:42.672936 10525 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:45791
I20260807 02:30:42.672963 10709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:45791 every 8 connection(s)
I20260807 02:30:42.673985 10525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:30:42.678009 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.676322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:55861)
0807 02:30:42.677088 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.677200 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":126,"threads_started":1}
I20260807 02:30:42.678185 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 10525
W20260807 02:30:42.678488 10710 heartbeater.cc:644] Failed to heartbeat to 127.17.53.62:35283 (0 consecutive failures): Network error: Failed to ping master at 127.17.53.62:35283: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
I20260807 02:30:42.679181 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.678931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:36437)
0807 02:30:42.679056 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.679121 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 02:30:42.679913 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:42.679645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:46057)
0807 02:30:42.679808 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:42.679857 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260807 02:30:42.680173 10710 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 02:30:43.323261 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.322738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:39035)
0807 02:30:43.323069 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:43.323137 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":84,"threads_started":1}
I20260807 02:30:43.483445 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.483014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:49715)
0807 02:30:43.483289 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:43.483343 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260807 02:30:43.599115 10725 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:43.599409 10725 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:43.600670 10725 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
Aug 07 02:30:43 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069842, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.2@KRBTEST.COM
Aug 07 02:30:43 dist-test-slave-g170 krb5kdc[8254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069842, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.3@KRBTEST.COM
I20260807 02:30:43.616106 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.601118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:38111 (local address 127.17.53.3:45791)
0807 02:30:43.601569 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:43.601588 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:43.601644 (+ 56us) server_negotiation.cc:407] Connection header received
0807 02:30:43.602661 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:43.602672 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:43.603186 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:43.603572 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:43.604722 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.605842 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607261 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.607724 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607913 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:43.611254 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:43.611259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:43.611278 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:43.611309 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:43.613390 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614003 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:43.614009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:43.614013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:43.614133 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614494 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:43.614498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:43.614500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:43.614945 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:43.615098 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:43.615661 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 02:30:43.615940 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":115,"threads_started":1}
I20260807 02:30:43.616092 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.601137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:45791 (local address 127.17.53.1:38111)
0807 02:30:43.601540 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:43.601561 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:43.601639 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:43.604128 (+ 2489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:43.604131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:43.604326 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:43.604516 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:43.604530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.606013 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.606019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:43.607080 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:43.607092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.607875 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:43.608116 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:43.608923 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:43.608935 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:43.611041 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:43.613522 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:43.613529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:43.613543 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:43.613862 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:43.614237 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:43.614242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:43.614392 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:43.615084 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:43.615091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:43.615324 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 02:30:43.615521 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 02:30:43.615890 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":123,"spinlock_wait_cycles":16768,"thread_start_us":88,"threads_started":1}
I20260807 02:30:43.616230 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.600987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39377 (local address 127.17.53.1:52253)
0807 02:30:43.601164 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:43.601179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:43.601290 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:43.603640 (+ 2350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:43.603646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:43.604035 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:43.604515 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:43.604529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.606075 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.606078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:43.607078 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:43.607089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.607894 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:43.608103 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:43.608910 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:43.608927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:43.610807 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:43.613703 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:43.613707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:43.613711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:43.613998 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:43.614418 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:43.614424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:43.614524 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:43.615290 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:43.615294 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:43.615404 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 02:30:43.615535 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 02:30:43.615999 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":199,"spinlock_wait_cycles":2560}
I20260807 02:30:43.616245 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.601223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:52253 (local address 127.17.53.2:39377)
0807 02:30:43.601623 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:43.601647 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:43.601682 (+ 35us) server_negotiation.cc:407] Connection header received
0807 02:30:43.602661 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:43.602672 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:43.603186 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:43.603572 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:43.604731 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.605887 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607218 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:43.607740 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:43.607935 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:43.611070 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:43.611075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:43.611089 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:43.611119 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:43.613549 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614166 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:43.614171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:43.614176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:43.614305 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:43.614663 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:43.614668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:43.614670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:43.615162 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:43.615327 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:43.615721 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 02:30:43.616042 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":113,"threads_started":1}
W20260807 02:30:43.619105 10138 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
W20260807 02:30:43.619345 10137 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:43.619508 10137 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8; no voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7
I20260807 02:30:43.619905 10725 raft_consensus.cc:2726] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:30:43.681282 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:43.680754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:56731)
0807 02:30:43.681092 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:43.681171 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":84,"threads_started":1}
I20260807 02:30:44.324976 10731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:44.324418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:36389)
0807 02:30:44.324728 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:44.324867 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":89,"threads_started":1}
I20260807 02:30:44.484687 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:44.484249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:35327)
0807 02:30:44.484535 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:44.484591 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":87,"threads_started":1}
I20260807 02:30:44.682735 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:44.682234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:34373)
0807 02:30:44.682546 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:44.682613 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
I20260807 02:30:45.326365 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:45.325887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:44325)
0807 02:30:45.326189 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:45.326263 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":81,"threads_started":1}
I20260807 02:30:45.486120 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:45.485669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:36249)
0807 02:30:45.485956 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:45.486012 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
I20260807 02:30:45.538542 10737 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:45.538754 10737 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:45.539220 10737 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
W20260807 02:30:45.540017 10138 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
W20260807 02:30:45.540177 10137 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:45.540256 10137 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8; no voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7
I20260807 02:30:45.540406 10737 raft_consensus.cc:2726] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:30:45.684161 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:45.683721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:48409)
0807 02:30:45.684004 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:45.684064 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
I20260807 02:30:46.328002 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:46.327335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:47335)
0807 02:30:46.327711 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:46.327860 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":89,"threads_started":1}
I20260807 02:30:46.487466 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:46.487011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:36587)
0807 02:30:46.487310 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:46.487367 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":110,"threads_started":1}
I20260807 02:30:46.685612 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:46.685112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:41813)
0807 02:30:46.685442 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:46.685511 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":86,"threads_started":1}
I20260807 02:30:47.329510 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:47.328997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:42315)
0807 02:30:47.329346 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:47.329410 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":80,"threads_started":1}
I20260807 02:30:47.488680 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:47.488251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:39743)
0807 02:30:47.488533 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:47.488587 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260807 02:30:47.550007 10750 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:47.550189 10750 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:47.550665 10750 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
W20260807 02:30:47.551590 10137 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
W20260807 02:30:47.551811 10138 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:47.551904 10138 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8; no voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7
I20260807 02:30:47.552107 10750 raft_consensus.cc:2726] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:30:47.686952 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:47.686483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:41341)
0807 02:30:47.686756 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:47.686842 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260807 02:30:48.330735 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:48.330241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:33439)
0807 02:30:48.330534 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:48.330588 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":93,"threads_started":1}
I20260807 02:30:48.489982 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:48.489493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:51527)
0807 02:30:48.489812 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:48.489870 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":79,"threads_started":1}
I20260807 02:30:48.688295 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:48.687854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:38135)
0807 02:30:48.688132 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:48.688190 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260807 02:30:49.332149 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:49.331620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:59205)
0807 02:30:49.331968 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:49.332038 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":96,"threads_started":1}
I20260807 02:30:49.491379 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:49.490823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:50655)
0807 02:30:49.491202 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:49.491273 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":105,"threads_started":1}
I20260807 02:30:49.597662 10758 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:49.597846 10758 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:49.598341 10758 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
W20260807 02:30:49.599296 10138 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
W20260807 02:30:49.599455 10137 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:49.599567 10137 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8; no voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7
I20260807 02:30:49.599723 10758 raft_consensus.cc:2726] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:30:49.689810 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:49.689260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:53185)
0807 02:30:49.689631 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:49.689703 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":84,"threads_started":1}
I20260807 02:30:50.333494 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:50.333046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:52375)
0807 02:30:50.333334 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:50.333396 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
I20260807 02:30:50.492723 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:50.492198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:37827)
0807 02:30:50.492516 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:50.492578 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260807 02:30:50.691340 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:50.690774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:42031)
0807 02:30:50.691164 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:50.691227 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":120,"threads_started":1}
I20260807 02:30:51.334885 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:51.334369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:44517)
0807 02:30:51.334704 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:51.334772 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":91,"threads_started":1}
I20260807 02:30:51.494153 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:51.493676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:50385)
0807 02:30:51.493978 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:51.494045 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260807 02:30:51.692593 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:51.692153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:59947)
0807 02:30:51.692430 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:51.692488 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
I20260807 02:30:52.336591 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:52.336101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:51289)
0807 02:30:52.336413 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:52.336483 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260807 02:30:52.495816 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:52.495310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:50261)
0807 02:30:52.495644 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:52.495704 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260807 02:30:52.693876 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:52.693448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:35919)
0807 02:30:52.693711 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:52.693768 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":80,"threads_started":1}
I20260807 02:30:53.337937 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:53.337503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:50473)
0807 02:30:53.337782 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:53.337838 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260807 02:30:53.497224 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:53.496714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:46281)
0807 02:30:53.497060 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:53.497121 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":166,"threads_started":1}
I20260807 02:30:53.695434 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:53.694909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:33047)
0807 02:30:53.695248 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:53.695324 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
I20260807 02:30:54.339375 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:54.338881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:57147)
0807 02:30:54.339184 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:54.339270 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260807 02:30:54.498943 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:54.498414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:40331)
0807 02:30:54.498741 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:54.498838 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
I20260807 02:30:54.697419 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:54.696630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:36689)
0807 02:30:54.697139 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:54.697233 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":222,"threads_started":1}
I20260807 02:30:55.340616 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:55.340174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:51787)
0807 02:30:55.340456 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:55.340510 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260807 02:30:55.500360 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:55.499887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:50549)
0807 02:30:55.500191 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:55.500257 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260807 02:30:55.698753 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:55.698257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:40717)
0807 02:30:55.698578 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:55.698640 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":110,"threads_started":1}
I20260807 02:30:56.341945 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:56.341494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:48635)
0807 02:30:56.341782 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:56.341840 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":131,"threads_started":1}
I20260807 02:30:56.501576 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:56.501142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:33659)
0807 02:30:56.501421 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:56.501478 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
I20260807 02:30:56.700026 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:56.699579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:36205)
0807 02:30:56.699863 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:56.699922 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260807 02:30:57.103941 10797 raft_consensus.cc:482] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:30:57.104086 10797 raft_consensus.cc:504] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f61d76f543f24d0e83086f9f2996e3e7" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 45791 } } peers { permanent_uuid: "4a318aeac5534af280f8050cbf9f3eb6" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39377 } } peers { permanent_uuid: "09eaae16e56d4a40b1a4d55d6a62a3f8" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 41617 } }
I20260807 02:30:57.104525 10797 leader_election.cc:288] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791), 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377)
W20260807 02:30:57.105449 10138 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f61d76f543f24d0e83086f9f2996e3e7 (127.17.53.3:45791): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
W20260807 02:30:57.105571 10137 leader_election.cc:341] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a318aeac5534af280f8050cbf9f3eb6 (127.17.53.2:39377): Not found: Tablet not found: 2eea3667c0a84522b723b13a8175cf3b
I20260807 02:30:57.105692 10138 leader_election.cc:302] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09eaae16e56d4a40b1a4d55d6a62a3f8; no voters: 4a318aeac5534af280f8050cbf9f3eb6, f61d76f543f24d0e83086f9f2996e3e7
I20260807 02:30:57.105906 10797 raft_consensus.cc:2726] T 2eea3667c0a84522b723b13a8175cf3b P 09eaae16e56d4a40b1a4d55d6a62a3f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 02:30:57.343580 10798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:57.343156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.1:54657)
0807 02:30:57.343423 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:57.343480 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260807 02:30:57.503027 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:57.502546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.2:41405)
0807 02:30:57.502879 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:57.502931 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":146,"threads_started":1}
I20260807 02:30:57.687079 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 10053
I20260807 02:30:57.701542 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:57.701094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35283 (local address 127.17.53.3:48385)
0807 02:30:57.701378 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:57.701441 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.62:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
I20260807 02:30:57.706804 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 10320
I20260807 02:30:57.723416 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 10525
2026-08-07T02:30:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59934 ms)
[----------] 30 tests from SecurityITest (255496 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:30:57 dist-test-slave-g170 krb5kdc[10805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:30:57 dist-test-slave-g170 krb5kdc[10805](info): set up 2 sockets
Aug 07 02:30:57 dist-test-slave-g170 krb5kdc[10805](info): commencing operation
krb5kdc: starting...
W20260807 02:30:59.005282 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:30:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:30:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:30:59.143536 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35907
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:42001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:59.264495 10823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:59.264868 10823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:59.265002 10823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:59.269250 10823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:30:59.269340 10823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:59.269384 10823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:59.269452 10823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:30:59.269495 10823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:30:59.274956 10823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:59.276098 10823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:59.277230 10823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:59.286316 10835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:59.286465 10823 server_base.cc:1034] running on GCE node
W20260807 02:30:59.286381 10839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:59.286369 10836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:59.287189 10823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:59.288326 10823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:59.289518 10823 hybrid_clock.cc:648] HybridClock initialized: now 1786069859289498 us; error 32 us; skew 500 ppm
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:59.292338 10823 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:30:59.293290 10823 webserver.cc:460] Webserver started at http://127.17.53.62:33811/ using document root <none> and password file <none>
I20260807 02:30:59.293807 10823 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:59.293867 10823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:59.294034 10823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:30:59.296031 10823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "48be8dd5e0704bb29dfe7282a09ea338"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8a23dc5d16a7d44c75a4cad79afa1cbb"
server_key_iv: "2877da06e2417057ff37399f0cff0cfe"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.296553 10823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "48be8dd5e0704bb29dfe7282a09ea338"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8a23dc5d16a7d44c75a4cad79afa1cbb"
server_key_iv: "2877da06e2417057ff37399f0cff0cfe"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.300340 10823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 02:30:59.303045 10846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.304454 10823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:30:59.304571 10823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "48be8dd5e0704bb29dfe7282a09ea338"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8a23dc5d16a7d44c75a4cad79afa1cbb"
server_key_iv: "2877da06e2417057ff37399f0cff0cfe"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.304656 10823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:59.318353 10823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:59.321305 10823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:59.321461 10823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:59.328948 10823 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35907
I20260807 02:30:59.328974 10921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35907 every 8 connection(s)
I20260807 02:30:59.329888 10823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:30:59.331202 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 10823
I20260807 02:30:59.331322 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:30:59.331665 17620 external_mini_cluster.cc:1428] Setting key a009f6773c8dfe665f8ee0fdb0d03691
I20260807 02:30:59.333268 10922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:30:59.340189 10922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: Bootstrap starting.
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:30:59.342952 10922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: Neither blocks nor log segments found. Creating new log.
I20260807 02:30:59.343923 10922 log.cc:824] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: Log is configured to *not* fsync() on all Append() calls
I20260807 02:30:59.346451 10922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: No bootstrap required, opened a new log
I20260807 02:30:59.347489 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:59.333179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.17.53.62:35907)
0807 02:30:59.333654 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:59.333673 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:59.333716 (+ 43us) server_negotiation.cc:407] Connection header received
0807 02:30:59.334539 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:59.334562 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:59.334986 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:59.335349 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:59.336046 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.337575 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.338454 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.339162 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.339343 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:59.341929 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:59.341935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:59.341951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:59.341983 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:59.344126 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.344696 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.344701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.344705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.344910 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.345232 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.345235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.345237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.345744 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:59.345940 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:59.346205 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 02:30:59.346524 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":165,"threads_started":1}
I20260807 02:30:59.350754 10922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } }
I20260807 02:30:59.350966 10922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:30:59.351009 10922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48be8dd5e0704bb29dfe7282a09ea338, State: Initialized, Role: FOLLOWER
I20260807 02:30:59.351419 10922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } }
I20260807 02:30:59.351541 10922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:30:59.351581 10922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:30:59.351699 10922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:30:59.352876 10922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } }
I20260807 02:30:59.353216 10922 leader_election.cc:302] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48be8dd5e0704bb29dfe7282a09ea338; no voters:
I20260807 02:30:59.353601 10922 leader_election.cc:288] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:30:59.353930 10930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:30:59.354215 10930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [term 1 LEADER]: Becoming Leader. State: Replica: 48be8dd5e0704bb29dfe7282a09ea338, State: Running, Role: LEADER
I20260807 02:30:59.354524 10930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } }
I20260807 02:30:59.354947 10922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:30:59.356164 10933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48be8dd5e0704bb29dfe7282a09ea338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } } }
I20260807 02:30:59.356299 10933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:59.356696 10943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:30:59.356686 10935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48be8dd5e0704bb29dfe7282a09ea338. Latest consensus state: current_term: 1 leader_uuid: "48be8dd5e0704bb29dfe7282a09ea338" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48be8dd5e0704bb29dfe7282a09ea338" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35907 } } }
I20260807 02:30:59.357411 10935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338 [sys.catalog]: This master's current role is: LEADER
I20260807 02:30:59.360013 10943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:30:59.367300 10943 catalog_manager.cc:1347] Generated new cluster ID: ecf1e1dca89440b19b6cc64eeda30adb
I20260807 02:30:59.367391 10943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:30:59.376310 10943 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:30:59.377260 10943 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:30:59.381943 10943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: Generated new TSK 0
I20260807 02:30:59.382606 10943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:30:59.427666 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:42001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:59.563460 10956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:59.563714 10956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:59.563759 10956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:59.567731 10956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:59.567786 10956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:59.567842 10956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:30:59.573071 10956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:59.574190 10956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:59.575174 10956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:30:59.585067 10956 server_base.cc:1034] running on GCE node
W20260807 02:30:59.585027 10964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:59.585088 10965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:59.585027 10967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:59.585688 10956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:59.586372 10956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:59.587595 10956 hybrid_clock.cc:648] HybridClock initialized: now 1786069859587563 us; error 42 us; skew 500 ppm
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:59.590816 10956 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:30:59.591751 10956 webserver.cc:460] Webserver started at http://127.17.53.1:37243/ using document root <none> and password file <none>
I20260807 02:30:59.592379 10956 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:59.592484 10956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:59.592844 10956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:30:59.595163 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "f5116f4ef57945fda0a9953a2bd7cd57"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8e127b2f8db859c47441c777ce99e22e"
server_key_iv: "baf0223fe86111ccd11b8304a345a349"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.595799 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "f5116f4ef57945fda0a9953a2bd7cd57"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8e127b2f8db859c47441c777ce99e22e"
server_key_iv: "baf0223fe86111ccd11b8304a345a349"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.600191 10956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 02:30:59.603132 10979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.604496 10956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:30:59.604645 10956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "f5116f4ef57945fda0a9953a2bd7cd57"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "8e127b2f8db859c47441c777ce99e22e"
server_key_iv: "baf0223fe86111ccd11b8304a345a349"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.604826 10956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:59.628248 10956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:59.632020 10956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:59.632257 10956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:59.632959 10956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:59.633895 10956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:59.633968 10956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.634043 10956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:59.634095 10956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.643746 10956 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:40281
I20260807 02:30:59.643764 11133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:40281 every 8 connection(s)
I20260807 02:30:59.644711 10956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:30:59.646071 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 10956
I20260807 02:30:59.646212 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:30:59.646565 17620 external_mini_cluster.cc:1428] Setting key a4385105a79273ee5e6bed5de4b3c804
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:30:59.660637 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:59.647007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49043 (local address 127.17.53.62:35907)
0807 02:30:59.647191 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:59.647195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:59.648157 (+ 962us) server_negotiation.cc:407] Connection header received
0807 02:30:59.649677 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:59.649683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:59.649749 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:59.649858 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:59.651081 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.651940 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.653194 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.653654 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.653780 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:59.656599 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:59.656603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:59.656608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:59.656639 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:59.658539 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.659101 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.659104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.659105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.659184 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.659598 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.659600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.659601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.659780 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:59.659859 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:59.660390 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 02:30:59.660511 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:30:59.661938 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:59.647475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35907 (local address 127.17.53.1:49043)
0807 02:30:59.648002 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:59.648063 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:59.649457 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:59.650022 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:59.650030 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:59.650420 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:59.650922 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:59.650936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.652078 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.652081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:59.653063 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:59.653073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.653822 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.653825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:59.653968 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:59.654676 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:59.654694 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:59.656416 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:59.658687 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:59.658692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:59.658704 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:59.659003 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:59.659326 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:59.659333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:59.659338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:59.659510 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:59.659879 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:59.659885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:59.660125 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 02:30:59.660293 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 02:30:59.660654 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":160,"threads_started":1}
I20260807 02:30:59.663388 11134 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35907
I20260807 02:30:59.663672 11134 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:59.664363 11134 heartbeater.cc:511] Master 127.17.53.62:35907 requested a full tablet report, sending...
I20260807 02:30:59.666312 10868 ts_manager.cc:194] Registered new tserver with Master: f5116f4ef57945fda0a9953a2bd7cd57 (127.17.53.1:40281)
I20260807 02:30:59.668062 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:49043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:30:59.680982 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:42001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:30:59.800370 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:30:59.800642 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:30:59.800686 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:30:59.804773 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:30:59.804826 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:30:59.804885 11146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:30:59.810344 11146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:30:59.811390 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:30:59.812422 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:30:59.822381 11160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:59.822475 11152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:30:59.822427 11154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:30:59.822969 11146 server_base.cc:1034] running on GCE node
I20260807 02:30:59.823391 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:30:59.824101 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:30:59.825364 11146 hybrid_clock.cc:648] HybridClock initialized: now 1786069859825328 us; error 58 us; skew 500 ppm
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:30:59.835589 11146 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:30:59.836612 11146 webserver.cc:460] Webserver started at http://127.17.53.2:34147/ using document root <none> and password file <none>
I20260807 02:30:59.837325 11146 fs_manager.cc:359] Metadata directory not provided
I20260807 02:30:59.837428 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:30:59.837687 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:30:59.839885 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "752478bf7889441abd341aa579f5e073"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "3ec4173e49e293d3859593e3974b6a11"
server_key_iv: "8b31fc3aed2a2e2868709159a4196187"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.840497 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "752478bf7889441abd341aa579f5e073"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "3ec4173e49e293d3859593e3974b6a11"
server_key_iv: "8b31fc3aed2a2e2868709159a4196187"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.844461 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:30:59.847268 11171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.848659 11146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:30:59.848843 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "752478bf7889441abd341aa579f5e073"
format_stamp: "Formatted at 2026-08-07 02:30:59 on dist-test-slave-g170"
server_key: "3ec4173e49e293d3859593e3974b6a11"
server_key_iv: "8b31fc3aed2a2e2868709159a4196187"
server_key_version: "encryptionkey@0"
I20260807 02:30:59.848975 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:30:59.861907 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:30:59.864877 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:30:59.865082 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:30:59.865713 11146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:30:59.866569 11146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:30:59.866641 11146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.866714 11146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:30:59.866767 11146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:30:59.875782 11146 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:40581
I20260807 02:30:59.875854 11354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:40581 every 8 connection(s)
I20260807 02:30:59.876701 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:30:59.879274 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 11146
I20260807 02:30:59.879415 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:30:59.879834 17620 external_mini_cluster.cc:1428] Setting key 14ee3d1463c8b9f9afbfb9c9bd61403b
Aug 07 02:30:59 dist-test-slave-g170 krb5kdc[10805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:30:59.892110 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:59.878586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:44401 (local address 127.17.53.62:35907)
0807 02:30:59.878769 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 02:30:59.878775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:30:59.879740 (+ 965us) server_negotiation.cc:407] Connection header received
0807 02:30:59.881471 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:30:59.881474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:30:59.881518 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:30:59.881585 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:30:59.883302 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.883827 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.885086 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.885310 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.885422 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:59.887960 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:30:59.887964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:30:59.887966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:30:59.887990 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:30:59.889790 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.890371 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.890378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.890380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.890475 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:30:59.890885 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:30:59.890890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:30:59.890892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:30:59.891110 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:30:59.891236 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 02:30:59.891770 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 02:30:59.891914 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:30:59.892997 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:30:59.878968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35907 (local address 127.17.53.2:44401)
0807 02:30:59.879523 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 02:30:59.879586 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 02:30:59.881191 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:30:59.881831 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:30:59.881844 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:30:59.882401 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:30:59.883120 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:59.883144 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.883961 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.883965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:59.884960 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:30:59.884970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:30:59.885464 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:30:59.885469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:30:59.885642 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:30:59.886281 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:30:59.886297 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:30:59.887789 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:30:59.889927 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:59.889935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:59.889950 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:59.890251 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:59.890593 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:30:59.890596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:30:59.890599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:30:59.890766 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:30:59.891237 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:30:59.891246 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:30:59.891506 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 02:30:59.891722 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 02:30:59.892072 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:30:59.894515 11355 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35907
I20260807 02:30:59.894821 11355 heartbeater.cc:464] Registering TS with master...
I20260807 02:30:59.895342 11355 heartbeater.cc:511] Master 127.17.53.62:35907 requested a full tablet report, sending...
I20260807 02:30:59.896556 10868 ts_manager.cc:194] Registered new tserver with Master: 752478bf7889441abd341aa579f5e073 (127.17.53.2:40581)
I20260807 02:30:59.897469 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:44401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:30:59.910198 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:42001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:00.028882 11362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:00.029198 11362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:00.029282 11362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:00.033339 11362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:00.033435 11362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:00.033528 11362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:00.038862 11362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:35907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:00.039937 11362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:00.041002 11362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:00.050902 11378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.050918 11380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.050910 11376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:00.051090 11362 server_base.cc:1034] running on GCE node
I20260807 02:31:00.051507 11362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:00.052248 11362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:00.053442 11362 hybrid_clock.cc:648] HybridClock initialized: now 1786069860053420 us; error 31 us; skew 500 ppm
Aug 07 02:31:00 dist-test-slave-g170 krb5kdc[10805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:00.056246 11362 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:31:00.057188 11362 webserver.cc:460] Webserver started at http://127.17.53.3:45515/ using document root <none> and password file <none>
I20260807 02:31:00.057695 11362 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:00.057753 11362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:00.057924 11362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:00.059868 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "275a1e41cd494fb9a1dbeb5b18f09c39"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "03057900cdfec0e0cbdf2de045a7ff27"
server_key_iv: "49cf8426c1a0b4b4cb2a150b71bad64c"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.060361 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "275a1e41cd494fb9a1dbeb5b18f09c39"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "03057900cdfec0e0cbdf2de045a7ff27"
server_key_iv: "49cf8426c1a0b4b4cb2a150b71bad64c"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.064015 11362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 02:31:00.066444 11392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.067772 11362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:31:00.067875 11362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "275a1e41cd494fb9a1dbeb5b18f09c39"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "03057900cdfec0e0cbdf2de045a7ff27"
server_key_iv: "49cf8426c1a0b4b4cb2a150b71bad64c"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.068013 11362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:00.090945 11362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:00.094094 11362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:00.094309 11362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:00.094934 11362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:00.095788 11362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:00.095836 11362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.095901 11362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:00.095934 11362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.105472 11362 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:46053
I20260807 02:31:00.105513 11556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:46053 every 8 connection(s)
I20260807 02:31:00.106393 11362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:00.107502 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 11362
I20260807 02:31:00.107662 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:00.108016 17620 external_mini_cluster.cc:1428] Setting key 292f532ae7d4eacae1f507ca6f8dd50d
Aug 07 02:31:00 dist-test-slave-g170 krb5kdc[10805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069860, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:00.120383 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.108543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:37273 (local address 127.17.53.62:35907)
0807 02:31:00.108778 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.108794 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.109908 (+ 1114us) server_negotiation.cc:407] Connection header received
0807 02:31:00.111292 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.111295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.111342 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.111406 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:00.112558 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.113141 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.114291 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.114494 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.114570 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.117216 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:00.117219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:00.117221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:00.117245 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:00.118604 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:00.119099 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:00.119101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:00.119103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:00.119177 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:00.119472 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:00.119475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:00.119476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:00.119644 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:00.119720 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.120173 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.120266 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:31:00.121300 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.109090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35907 (local address 127.17.53.3:37273)
0807 02:31:00.109723 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.109785 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.111091 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.111575 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.111581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.111943 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:00.112433 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.112446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.113257 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.113260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.114178 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.114188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.114621 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.114624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.114803 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.115448 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:00.115464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:00.117062 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:00.118725 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:00.118731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:00.118745 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:00.119006 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:00.119257 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:00.119259 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:00.119261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:00.119394 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:00.119739 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:00.119744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:00.119967 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.120125 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.120391 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":163,"threads_started":1}
I20260807 02:31:00.122601 11557 heartbeater.cc:347] Connected to a master server at 127.17.53.62:35907
I20260807 02:31:00.122816 11557 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:00.123260 11557 heartbeater.cc:511] Master 127.17.53.62:35907 requested a full tablet report, sending...
I20260807 02:31:00.124218 10868 ts_manager.cc:194] Registered new tserver with Master: 275a1e41cd494fb9a1dbeb5b18f09c39 (127.17.53.3:46053)
I20260807 02:31:00.124747 10868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:37273
I20260807 02:31:00.134274 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:00.144101 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.136402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.17.53.62:35907)
0807 02:31:00.136504 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.136508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.136794 (+ 286us) server_negotiation.cc:407] Connection header received
0807 02:31:00.136912 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.136915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.136966 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.137049 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:00.137775 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.138280 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.139150 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.139377 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.139486 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.140974 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:00.140978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:00.140980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:00.141004 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:00.142333 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:00.142779 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:00.142789 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:00.142791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:00.142870 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:00.143156 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:00.143159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:00.143161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:00.143390 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:00.143538 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.143804 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.143930 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 02:31:00.147526 10868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:00.150986 10868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:00.166901 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.160467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.17.53.1:40281)
0807 02:31:00.160887 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.160908 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.160937 (+ 29us) server_negotiation.cc:407] Connection header received
0807 02:31:00.161216 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.161224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.161462 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.161630 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:00.162222 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.163561 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.165103 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.165887 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.165955 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.166063 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.166602 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.166709 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":68,"threads_started":1}
I20260807 02:31:00.167106 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.161135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:46053 (local address 127.0.0.1:40546)
0807 02:31:00.161612 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.161626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.161708 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.162620 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.162623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.162640 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:00.162736 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.162741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.163743 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.163747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.165186 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.165198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.166566 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.166570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.166693 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.166720 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.166796 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.166934 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":101,"thread_start_us":104,"threads_started":1}
I20260807 02:31:00.167634 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.160330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:40281 (local address 127.0.0.1:49486)
0807 02:31:00.160810 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.160857 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.160985 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.161846 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.161853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.161880 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:00.162040 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.162048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.163698 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.163704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.164958 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.164970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.166267 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.166271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.166400 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.166441 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.167417 (+ 976us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.167499 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260807 02:31:00.167778 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.161188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.17.53.2:40581)
0807 02:31:00.161940 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.161951 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.161995 (+ 44us) server_negotiation.cc:407] Connection header received
0807 02:31:00.162073 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.162079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.162271 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.162419 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:00.162884 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.164130 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.165600 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.166582 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.166693 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.166851 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.167532 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.167623 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":166,"threads_started":1}
I20260807 02:31:00.167944 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.160408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40581 (local address 127.0.0.1:59276)
0807 02:31:00.161336 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.161353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.161446 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.162404 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.162407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.162429 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:00.162536 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.162544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.164286 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.164289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.165462 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.165472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.166920 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.166924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.167022 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.167045 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.167125 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.167221 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":25,"thread_start_us":119,"threads_started":1}
I20260807 02:31:00.168332 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.161761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.17.53.3:46053)
0807 02:31:00.162148 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.162167 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.162186 (+ 19us) server_negotiation.cc:407] Connection header received
0807 02:31:00.162256 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.162260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.162430 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.162567 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:00.162851 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.163592 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.165332 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.166206 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.166332 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.166490 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.167466 (+ 976us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.167581 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":165,"threads_started":1}
I20260807 02:31:00.171311 11463 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 (DEFAULT_TABLE table=test-table [id=e3c7991a259442caac3a9a8e26321340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:00.171345 11041 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 (DEFAULT_TABLE table=test-table [id=e3c7991a259442caac3a9a8e26321340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:00.171514 11251 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 (DEFAULT_TABLE table=test-table [id=e3c7991a259442caac3a9a8e26321340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:00.172429 11463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab567d8ad3140d2ad704d054e1c8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:00.172511 11041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab567d8ad3140d2ad704d054e1c8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:00.172511 11251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab567d8ad3140d2ad704d054e1c8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:00.179899 11576 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Bootstrap starting.
I20260807 02:31:00.181185 11578 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Bootstrap starting.
I20260807 02:31:00.182783 11576 tablet_bootstrap.cc:654] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:00.184108 11578 tablet_bootstrap.cc:654] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:00.184372 11576 log.cc:824] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:00.185030 11578 log.cc:824] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:00.185119 11577 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Bootstrap starting.
I20260807 02:31:00.186790 11576 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: No bootstrap required, opened a new log
I20260807 02:31:00.187026 11576 ts_tablet_manager.cc:1397] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 02:31:00.187054 11577 tablet_bootstrap.cc:654] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:00.187227 11578 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: No bootstrap required, opened a new log
I20260807 02:31:00.187435 11578 ts_tablet_manager.cc:1397] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:31:00.187809 11577 log.cc:824] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:00.190016 11577 tablet_bootstrap.cc:492] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: No bootstrap required, opened a new log
I20260807 02:31:00.190217 11577 ts_tablet_manager.cc:1397] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:31:00.191661 11578 raft_consensus.cc:348] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.191656 11576 raft_consensus.cc:348] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.191972 11576 raft_consensus.cc:374] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:00.191972 11578 raft_consensus.cc:374] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:00.192042 11576 raft_consensus.cc:729] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 275a1e41cd494fb9a1dbeb5b18f09c39, State: Initialized, Role: FOLLOWER
I20260807 02:31:00.192041 11578 raft_consensus.cc:729] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752478bf7889441abd341aa579f5e073, State: Initialized, Role: FOLLOWER
I20260807 02:31:00.192550 11576 consensus_queue.cc:260] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.192550 11578 consensus_queue.cc:260] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.193511 11557 heartbeater.cc:503] Master 127.17.53.62:35907 was elected leader, sending a full tablet report...
I20260807 02:31:00.193696 11578 ts_tablet_manager.cc:1428] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 02:31:00.193696 11576 ts_tablet_manager.cc:1428] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260807 02:31:00.193972 11355 heartbeater.cc:503] Master 127.17.53.62:35907 was elected leader, sending a full tablet report...
I20260807 02:31:00.194233 11577 raft_consensus.cc:348] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.194459 11577 raft_consensus.cc:374] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:00.194515 11577 raft_consensus.cc:729] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5116f4ef57945fda0a9953a2bd7cd57, State: Initialized, Role: FOLLOWER
I20260807 02:31:00.195828 11577 consensus_queue.cc:260] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.196587 11134 heartbeater.cc:503] Master 127.17.53.62:35907 was elected leader, sending a full tablet report...
I20260807 02:31:00.198077 11577 ts_tablet_manager.cc:1428] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 02:31:00.337816 11587 raft_consensus.cc:482] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:00.338078 11587 raft_consensus.cc:504] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.339540 11587 leader_election.cc:288] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 752478bf7889441abd341aa579f5e073 (127.17.53.2:40581), 275a1e41cd494fb9a1dbeb5b18f09c39 (127.17.53.3:46053)
I20260807 02:31:00.344498 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.340050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:48907 (local address 127.17.53.3:46053)
0807 02:31:00.340207 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.340214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.340446 (+ 232us) server_negotiation.cc:407] Connection header received
0807 02:31:00.340671 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.340674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.340752 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.340858 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:00.341302 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.342163 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.343511 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.343926 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.344049 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.344122 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.344303 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.344381 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:31:00.344527 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.339796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:46053 (local address 127.17.53.1:48907)
0807 02:31:00.340317 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.340336 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.340484 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.340933 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.340938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.340959 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:00.341072 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.341099 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.342308 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.342312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.343379 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.343387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.344055 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.344059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.344163 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.344190 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.344285 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.344360 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":116,"thread_start_us":92,"threads_started":1}
I20260807 02:31:00.344702 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.340260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:47979 (local address 127.17.53.2:40581)
0807 02:31:00.340429 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.340435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.340456 (+ 21us) server_negotiation.cc:407] Connection header received
0807 02:31:00.340595 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.340598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.340674 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.340779 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:00.341152 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.341741 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.343221 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.343958 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.344138 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.344224 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.344441 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.344528 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 02:31:00.345098 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.339788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40581 (local address 127.17.53.1:47979)
0807 02:31:00.340258 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.340274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.340451 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.340821 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.340824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.340846 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:00.341006 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.341014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.341863 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.341868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.343062 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.343073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.344132 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.344137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.344273 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.344309 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.344422 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.344508 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":1,"thread_start_us":143,"threads_started":1}
I20260807 02:31:00.345268 11492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab567d8ad3140d2ad704d054e1c8632" candidate_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" is_pre_election: true
I20260807 02:31:00.345564 11492 raft_consensus.cc:2454] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5116f4ef57945fda0a9953a2bd7cd57 in term 0.
I20260807 02:31:00.345875 11283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab567d8ad3140d2ad704d054e1c8632" candidate_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "752478bf7889441abd341aa579f5e073" is_pre_election: true
I20260807 02:31:00.346136 10981 leader_election.cc:302] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 275a1e41cd494fb9a1dbeb5b18f09c39, f5116f4ef57945fda0a9953a2bd7cd57; no voters:
I20260807 02:31:00.346182 11283 raft_consensus.cc:2454] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5116f4ef57945fda0a9953a2bd7cd57 in term 0.
I20260807 02:31:00.346421 11587 raft_consensus.cc:2781] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:00.346544 11587 raft_consensus.cc:482] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:00.346680 11587 raft_consensus.cc:3037] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:00.347926 11587 raft_consensus.cc:504] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.348351 11587 leader_election.cc:288] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [CANDIDATE]: Term 1 election: Requested vote from peers 752478bf7889441abd341aa579f5e073 (127.17.53.2:40581), 275a1e41cd494fb9a1dbeb5b18f09c39 (127.17.53.3:46053)
I20260807 02:31:00.348852 11492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab567d8ad3140d2ad704d054e1c8632" candidate_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39"
I20260807 02:31:00.348964 11492 raft_consensus.cc:3037] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:00.350088 11492 raft_consensus.cc:2454] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5116f4ef57945fda0a9953a2bd7cd57 in term 1.
I20260807 02:31:00.350404 11283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ab567d8ad3140d2ad704d054e1c8632" candidate_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "752478bf7889441abd341aa579f5e073"
I20260807 02:31:00.350489 10981 leader_election.cc:302] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 275a1e41cd494fb9a1dbeb5b18f09c39, f5116f4ef57945fda0a9953a2bd7cd57; no voters:
I20260807 02:31:00.350536 11283 raft_consensus.cc:3037] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:00.350734 11587 raft_consensus.cc:2781] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:00.351099 11587 raft_consensus.cc:686] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [term 1 LEADER]: Becoming Leader. State: Replica: f5116f4ef57945fda0a9953a2bd7cd57, State: Running, Role: LEADER
I20260807 02:31:00.351486 11587 consensus_queue.cc:237] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } }
I20260807 02:31:00.351999 11283 raft_consensus.cc:2454] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5116f4ef57945fda0a9953a2bd7cd57 in term 1.
I20260807 02:31:00.355152 10861 catalog_manager.cc:5495] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5116f4ef57945fda0a9953a2bd7cd57 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5116f4ef57945fda0a9953a2bd7cd57" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 40281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 } health_report { overall_health: UNKNOWN } } }
W20260807 02:31:00.357337 11558 tablet.cc:2369] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:31:00.377902 11357 tablet.cc:2369] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:31:00.396454 11138 tablet.cc:2369] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:00.409590 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.405531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49500 (local address 127.17.53.1:40281)
0807 02:31:00.405671 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.405675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.405690 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:31:00.405829 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.405831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.405889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.405978 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:00.406178 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.406693 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.407822 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.408039 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.408187 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.408545 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:00.409110 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:00.409258 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.409362 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.409414 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:31:00.415745 11283 raft_consensus.cc:1261] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 1 FOLLOWER]: Refusing update from remote peer f5116f4ef57945fda0a9953a2bd7cd57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:00.415741 11492 raft_consensus.cc:1261] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 1 FOLLOWER]: Refusing update from remote peer f5116f4ef57945fda0a9953a2bd7cd57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:00.416581 11590 consensus_queue.cc:1035] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "275a1e41cd494fb9a1dbeb5b18f09c39" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 46053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:00.416786 11587 consensus_queue.cc:1035] T 0ab567d8ad3140d2ad704d054e1c8632 P f5116f4ef57945fda0a9953a2bd7cd57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "752478bf7889441abd341aa579f5e073" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 40581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:00.433014 11597 mvcc.cc:205] Tried to move back new op lower bound from 7315742148255047680 to 7315742148003160064. Current Snapshot: MvccSnapshot[applied={T|T < 7315742148255047680 or (T in {7315742148255047680})}]
I20260807 02:31:00.434702 10866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:00.434945 10866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:00.437970 10866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48be8dd5e0704bb29dfe7282a09ea338: Sending DeleteTablet for 3 replicas of tablet 0ab567d8ad3140d2ad704d054e1c8632
I20260807 02:31:00.438884 11251 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:59276
I20260807 02:31:00.439086 11463 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:40546
I20260807 02:31:00.439679 11610 tablet_replica.cc:330] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: stopping tablet replica
I20260807 02:31:00.440048 11041 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab567d8ad3140d2ad704d054e1c8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:00 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:49486
I20260807 02:31:00.440232 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 10956
I20260807 02:31:00.440155 11610 raft_consensus.cc:2229] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:00.440718 11610 raft_consensus.cc:2258] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:00.450925 11610 ts_tablet_manager.cc:1905] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:00.454504 11610 ts_tablet_manager.cc:1918] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:00.454634 11610 log.cc:1196] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/0ab567d8ad3140d2ad704d054e1c8632
I20260807 02:31:00.455054 11610 ts_tablet_manager.cc:1939] T 0ab567d8ad3140d2ad704d054e1c8632 P 275a1e41cd494fb9a1dbeb5b18f09c39: Deleting consensus metadata
I20260807 02:31:00.456859 11609 tablet_replica.cc:330] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073: stopping tablet replica
I20260807 02:31:00.456889 10848 catalog_manager.cc:4841] TS 275a1e41cd494fb9a1dbeb5b18f09c39 (127.17.53.3:46053): tablet 0ab567d8ad3140d2ad704d054e1c8632 (table test-table [id=e3c7991a259442caac3a9a8e26321340]) successfully deleted
I20260807 02:31:00.457213 11609 raft_consensus.cc:2229] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:00.457561 11609 raft_consensus.cc:2258] T 0ab567d8ad3140d2ad704d054e1c8632 P 752478bf7889441abd341aa579f5e073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:00.460734 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 11146
W20260807 02:31:00.465387 10848 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:40281 (error 108)
I20260807 02:31:00.467388 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.467028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:40281 (local address 127.0.0.1:49510)
0807 02:31:00.467193 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.467293 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:40281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 02:31:00.467672 10848 catalog_manager.cc:4584] TS f5116f4ef57945fda0a9953a2bd7cd57 (127.17.53.1:40281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ab567d8ad3140d2ad704d054e1c8632: Network error: Client connection negotiation failed: client connection to 127.17.53.1:40281: connect: Connection refused (error 111)
I20260807 02:31:00.485356 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 11362
I20260807 02:31:00.485692 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.485370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:40581 (local address 127.0.0.1:59292)
0807 02:31:00.485518 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.485586 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.2:40581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 02:31:00.497963 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 10823
2026-08-07T02:31:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2760 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T02:31:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:00Z Disabled control of system clock
I20260807 02:31:00.538147 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:33743
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:33743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:00.653002 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:00.653264 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:00.653301 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:00.657240 11618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:00.657296 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:00.657312 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:00.657328 11618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:00.657343 11618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:00.662465 11618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:33743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:33743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:00.663640 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:00.665174 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:00.673952 11629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.674003 11625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.674007 11632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:00.674216 11618 server_base.cc:1034] running on GCE node
I20260807 02:31:00.674906 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:00.676071 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:00.677270 11618 hybrid_clock.cc:648] HybridClock initialized: now 1786069860677262 us; error 32 us; skew 500 ppm
I20260807 02:31:00.679426 11618 webserver.cc:460] Webserver started at http://127.17.53.62:40961/ using document root <none> and password file <none>
I20260807 02:31:00.680095 11618 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:00.680199 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:00.680452 11618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:00.682608 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "40e3a3c4cde14910ba59658ec22c23b1"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "a58f7a823152640a72231a162a9b2fea"
server_key_iv: "e4e3329ed4dc4ba840b23e177c2f23d2"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.683193 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "40e3a3c4cde14910ba59658ec22c23b1"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "a58f7a823152640a72231a162a9b2fea"
server_key_iv: "e4e3329ed4dc4ba840b23e177c2f23d2"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.687433 11618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:00.690060 11641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.691319 11618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:31:00.691468 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "40e3a3c4cde14910ba59658ec22c23b1"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "a58f7a823152640a72231a162a9b2fea"
server_key_iv: "e4e3329ed4dc4ba840b23e177c2f23d2"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.691582 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:00.720391 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:00.721243 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:00.721468 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:00.728863 11618 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:33743
I20260807 02:31:00.728896 11724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:33743 every 8 connection(s)
I20260807 02:31:00.729867 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:00.733383 11725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:00.734964 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 11618
I20260807 02:31:00.735103 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:00.735457 17620 external_mini_cluster.cc:1428] Setting key 8fa550a81b784e205809303c00b105c0
I20260807 02:31:00.740108 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: Bootstrap starting.
I20260807 02:31:00.742523 11725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:00.743412 11725 log.cc:824] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:00.745275 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.737050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.17.53.62:33743)
0807 02:31:00.737828 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.737846 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.737885 (+ 39us) server_negotiation.cc:407] Connection header received
0807 02:31:00.738568 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.738587 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.738988 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.739357 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:00.739807 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.740886 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.741780 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.742250 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.742427 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.743321 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:00.743326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:00.743341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:00.743779 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:00.743834 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:00.743855 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:00.743965 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.744115 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.744368 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":148,"threads_started":1}
I20260807 02:31:00.746047 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: No bootstrap required, opened a new log
I20260807 02:31:00.749974 11725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } }
I20260807 02:31:00.750245 11725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:00.750337 11725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e3a3c4cde14910ba59658ec22c23b1, State: Initialized, Role: FOLLOWER
I20260807 02:31:00.750808 11725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } }
I20260807 02:31:00.750983 11725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:00.751053 11725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:00.751206 11725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:00.752229 11725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } }
I20260807 02:31:00.752580 11725 leader_election.cc:302] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40e3a3c4cde14910ba59658ec22c23b1; no voters:
I20260807 02:31:00.752959 11725 leader_election.cc:288] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:00.753077 11732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:00.753312 11732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [term 1 LEADER]: Becoming Leader. State: Replica: 40e3a3c4cde14910ba59658ec22c23b1, State: Running, Role: LEADER
I20260807 02:31:00.753654 11732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } }
I20260807 02:31:00.754235 11725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:00.755460 11734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40e3a3c4cde14910ba59658ec22c23b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } } }
I20260807 02:31:00.755605 11734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:00.755921 11735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40e3a3c4cde14910ba59658ec22c23b1. Latest consensus state: current_term: 1 leader_uuid: "40e3a3c4cde14910ba59658ec22c23b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e3a3c4cde14910ba59658ec22c23b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 33743 } } }
I20260807 02:31:00.756026 11746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:00.756042 11735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:00.758975 11746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:00.764099 11746 catalog_manager.cc:1347] Generated new cluster ID: dfb613a3b1064f38ae73668a924a1302
I20260807 02:31:00.764169 11746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:00.777006 11746 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:00.778227 11746 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:00.784282 11746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: Generated new TSK 0
I20260807 02:31:00.785097 11746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:00.794559 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--builtin_ntp_servers=127.17.53.20:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:00.913820 11759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:00.914137 11759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:00.914216 11759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:00.918469 11759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:00.918563 11759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:00.918665 11759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:31:00.923472 11759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:00.924612 11759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:00.926119 11759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:00.936121 11770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.936136 11775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:00.936105 11771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:00.936578 11759 server_base.cc:1034] running on GCE node
I20260807 02:31:00.937037 11759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:00.937729 11759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:00.938931 11759 hybrid_clock.cc:648] HybridClock initialized: now 1786069860938915 us; error 29 us; skew 500 ppm
I20260807 02:31:00.940912 11759 webserver.cc:460] Webserver started at http://127.17.53.1:34181/ using document root <none> and password file <none>
I20260807 02:31:00.941566 11759 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:00.941665 11759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:00.941887 11759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:00.943962 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "4fbc32be152046e7867e165449bc6519"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "0035368dfd00ea026e616178d4f05963"
server_key_iv: "672924af9066de4cd5d37ae53d9489e5"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.944541 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "4fbc32be152046e7867e165449bc6519"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "0035368dfd00ea026e616178d4f05963"
server_key_iv: "672924af9066de4cd5d37ae53d9489e5"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.948597 11759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 02:31:00.951437 11784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.952903 11759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:00.953045 11759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "4fbc32be152046e7867e165449bc6519"
format_stamp: "Formatted at 2026-08-07 02:31:00 on dist-test-slave-g170"
server_key: "0035368dfd00ea026e616178d4f05963"
server_key_iv: "672924af9066de4cd5d37ae53d9489e5"
server_key_version: "encryptionkey@0"
I20260807 02:31:00.953169 11759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:00.963858 11759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:00.964555 11759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:00.964797 11759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:00.965407 11759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:00.966427 11759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:00.966503 11759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.966580 11759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:00.966622 11759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:00.976742 11759 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:45691
I20260807 02:31:00.976789 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:45691 every 8 connection(s)
I20260807 02:31:00.977710 11759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:31:00.982388 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 11759
I20260807 02:31:00.982527 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:31:00.982906 17620 external_mini_cluster.cc:1428] Setting key 2a1f1ca7d72ac028444b4b52feda7349
I20260807 02:31:00.985878 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--builtin_ntp_servers=127.17.53.20:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:00.991046 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.979932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:46677 (local address 127.17.53.62:33743)
0807 02:31:00.980089 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:00.980093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:00.981359 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 02:31:00.983335 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:00.983339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:00.983407 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:00.983504 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:00.985331 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.986204 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.988218 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.988665 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.988858 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.990405 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:00.990409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:00.990412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:00.990429 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:00.990453 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:00.990463 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:00.990538 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:00.990809 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 02:31:00.990894 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:31:00.992323 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:00.980237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:33743 (local address 127.17.53.1:46677)
0807 02:31:00.981176 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:00.981222 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:00.983066 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:00.984115 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:00.984127 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:00.984252 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:00.985078 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.985103 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.986439 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.986445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.988002 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:00.988021 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:00.988879 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:00.988884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:00.989103 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:00.989999 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:00.990004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:00.990063 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:00.990178 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:00.990581 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:00.990591 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:00.990597 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:31:00.990785 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 02:31:00.991099 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":130,"threads_started":1}
I20260807 02:31:00.993705 11936 heartbeater.cc:347] Connected to a master server at 127.17.53.62:33743
I20260807 02:31:00.994078 11936 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:00.995061 11936 heartbeater.cc:511] Master 127.17.53.62:33743 requested a full tablet report, sending...
I20260807 02:31:00.997187 11667 ts_manager.cc:194] Registered new tserver with Master: 4fbc32be152046e7867e165449bc6519 (127.17.53.1:45691)
I20260807 02:31:00.999018 11667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.1:46677
W20260807 02:31:01.113718 11940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:01.114017 11940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:01.114089 11940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:01.118083 11940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:01.118158 11940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:01.118249 11940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:31:01.122929 11940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:01.124037 11940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:01.125530 11940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:01.135629 11949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:01.135694 11952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:01.135627 11950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:01.136016 11940 server_base.cc:1034] running on GCE node
I20260807 02:31:01.136541 11940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:01.137365 11940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:01.138612 11940 hybrid_clock.cc:648] HybridClock initialized: now 1786069861138595 us; error 26 us; skew 500 ppm
I20260807 02:31:01.140607 11940 webserver.cc:460] Webserver started at http://127.17.53.2:33739/ using document root <none> and password file <none>
I20260807 02:31:01.141287 11940 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:01.141389 11940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:01.141633 11940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:01.143777 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "7e21e1fe70354cd3bdce8e0953795e39"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "6565f87b1fc1de776f63ef2e1c8ba80b"
server_key_iv: "e0f846dfd12d13c7ab60bf7acb5daf02"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.144369 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "7e21e1fe70354cd3bdce8e0953795e39"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "6565f87b1fc1de776f63ef2e1c8ba80b"
server_key_iv: "e0f846dfd12d13c7ab60bf7acb5daf02"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.148170 11940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:01.150818 11961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.152441 11940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:31:01.152604 11940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "7e21e1fe70354cd3bdce8e0953795e39"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "6565f87b1fc1de776f63ef2e1c8ba80b"
server_key_iv: "e0f846dfd12d13c7ab60bf7acb5daf02"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.152738 11940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:01.169449 11940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:01.170239 11940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:01.170469 11940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:01.171139 11940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:01.172209 11940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:01.172290 11940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.172374 11940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:01.172423 11940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.183027 11940 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:38847
I20260807 02:31:01.183094 12154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:38847 every 8 connection(s)
I20260807 02:31:01.184084 11940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:31:01.185968 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 11940
I20260807 02:31:01.186112 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:31:01.186472 17620 external_mini_cluster.cc:1428] Setting key 4f4fd25135ebf45d4549c50436a18221
I20260807 02:31:01.188820 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--builtin_ntp_servers=127.17.53.20:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:01.195215 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.186359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:50895 (local address 127.17.53.62:33743)
0807 02:31:01.186536 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.186543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.187909 (+ 1366us) server_negotiation.cc:407] Connection header received
0807 02:31:01.189782 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.189786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.189848 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.189970 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:01.190971 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.191546 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.192878 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.193195 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.193324 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.194611 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:01.194615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:01.194618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:01.194632 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:01.194662 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:01.194671 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:01.194770 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.194946 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.195050 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:31:01.196185 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.186933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:33743 (local address 127.17.53.2:50895)
0807 02:31:01.187753 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.187811 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.189510 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.190171 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.190179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.190279 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:01.190809 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.190824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.191688 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.191692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.192676 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.192686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.193388 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.193394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.193573 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.194319 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:01.194321 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:01.194364 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:01.194435 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:01.194772 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:01.194786 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:01.194794 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.194945 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.195293 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":160,"threads_started":1}
I20260807 02:31:01.197554 12156 heartbeater.cc:347] Connected to a master server at 127.17.53.62:33743
I20260807 02:31:01.197893 12156 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:01.198513 12156 heartbeater.cc:511] Master 127.17.53.62:33743 requested a full tablet report, sending...
I20260807 02:31:01.199831 11667 ts_manager.cc:194] Registered new tserver with Master: 7e21e1fe70354cd3bdce8e0953795e39 (127.17.53.2:38847)
I20260807 02:31:01.200516 11667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.2:50895
W20260807 02:31:01.313179 12160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:01.313454 12160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:01.313525 12160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:01.317492 12160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:01.317560 12160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:01.317646 12160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:01.322445 12160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:33743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:01.323534 12160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:01.324996 12160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:01.335235 12165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:01.335268 12169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:01.335487 12160 server_base.cc:1034] running on GCE node
W20260807 02:31:01.335244 12166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:01.336061 12160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:01.336694 12160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:01.337956 12160 hybrid_clock.cc:648] HybridClock initialized: now 1786069861337921 us; error 48 us; skew 500 ppm
I20260807 02:31:01.339984 12160 webserver.cc:460] Webserver started at http://127.17.53.3:42327/ using document root <none> and password file <none>
I20260807 02:31:01.340653 12160 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:01.340775 12160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:01.341054 12160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:01.343333 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "6c68be41519d46478dc70b2f4efe4fa4"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "812b600c8009e9e234425b424994523b"
server_key_iv: "6a61134f5e6c75d46331b8a879218571"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.343976 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "6c68be41519d46478dc70b2f4efe4fa4"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "812b600c8009e9e234425b424994523b"
server_key_iv: "6a61134f5e6c75d46331b8a879218571"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.348074 12160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 02:31:01.350884 12180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.352391 12160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:31:01.352545 12160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "6c68be41519d46478dc70b2f4efe4fa4"
format_stamp: "Formatted at 2026-08-07 02:31:01 on dist-test-slave-g170"
server_key: "812b600c8009e9e234425b424994523b"
server_key_iv: "6a61134f5e6c75d46331b8a879218571"
server_key_version: "encryptionkey@0"
I20260807 02:31:01.352667 12160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:01.379833 12160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:01.380615 12160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:01.380875 12160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:01.381596 12160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:01.382738 12160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:01.382792 12160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.382836 12160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:01.382853 12160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:01.392946 12160 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:42287
I20260807 02:31:01.392982 12349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:42287 every 8 connection(s)
I20260807 02:31:01.393889 12160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:01.397315 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 12160
I20260807 02:31:01.397444 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:01.397769 17620 external_mini_cluster.cc:1428] Setting key ab014a26aa23c3c81e68716863be7811
I20260807 02:31:01.404594 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.396071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:54191 (local address 127.17.53.62:33743)
0807 02:31:01.396211 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.396215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.397464 (+ 1249us) server_negotiation.cc:407] Connection header received
0807 02:31:01.399282 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.399285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.399384 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.399474 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:01.400508 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.401094 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.402243 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.402468 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.402550 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.403655 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:01.403658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:01.403660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:01.403671 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:01.403689 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:01.403696 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:01.403755 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.404204 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.404265 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 02:31:01.405362 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.396431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:33743 (local address 127.17.53.3:54191)
0807 02:31:01.397262 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.397321 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.399028 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.399632 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.399639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.399731 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:01.400379 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.400394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.401229 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.401232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.402120 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.402130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.402603 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.402606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.402752 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.403413 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:01.403415 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:01.403455 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:01.403516 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:01.404001 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:01.404019 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:01.404025 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.404179 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.404437 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":144,"threads_started":1}
I20260807 02:31:01.406731 12350 heartbeater.cc:347] Connected to a master server at 127.17.53.62:33743
I20260807 02:31:01.406993 12350 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:01.407547 12350 heartbeater.cc:511] Master 127.17.53.62:33743 requested a full tablet report, sending...
I20260807 02:31:01.408849 11667 ts_manager.cc:194] Registered new tserver with Master: 6c68be41519d46478dc70b2f4efe4fa4 (127.17.53.3:42287)
I20260807 02:31:01.409490 11667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.3:54191
I20260807 02:31:01.412362 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:01.418645 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.414551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.17.53.62:33743)
0807 02:31:01.414703 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.414708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.414847 (+ 139us) server_negotiation.cc:407] Connection header received
0807 02:31:01.414956 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.414958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.414996 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.415061 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:01.415351 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.415848 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.416703 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.417148 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.417254 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.418181 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:01.418184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:01.418186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:01.418197 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:01.418217 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:01.418225 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:01.418290 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.418412 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.418460 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 02:31:01.421823 11667 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:01.424852 11667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:01.439924 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.434405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.17.53.3:42287)
0807 02:31:01.434896 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.434908 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.435391 (+ 483us) server_negotiation.cc:407] Connection header received
0807 02:31:01.435466 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.435471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.435614 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.435747 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:01.436180 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.437293 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.438670 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439219 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.439346 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.439553 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.439621 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.439732 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":154,"threads_started":1}
I20260807 02:31:01.439936 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.433651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:38847 (local address 127.0.0.1:40052)
0807 02:31:01.434471 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.434524 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.434652 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.435734 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.435736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.435756 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:01.435873 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.435879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.436990 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.436995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.438317 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.438329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439423 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.439427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.439522 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.439560 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.439614 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.439770 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":66,"thread_start_us":86,"threads_started":1}
I20260807 02:31:01.439986 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.433813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45691 (local address 127.0.0.1:53416)
0807 02:31:01.434909 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.434922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.435000 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.435515 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.435521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.435545 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:01.435720 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.435728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.437248 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.437254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.438583 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.438591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439615 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.439620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.439714 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.439740 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.439799 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.439860 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":106,"thread_start_us":135,"threads_started":1}
I20260807 02:31:01.439935 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.434062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42287 (local address 127.0.0.1:46722)
0807 02:31:01.435190 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.435204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.435291 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.435905 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.435908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.435924 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:01.436046 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.436053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.437413 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.437416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.438535 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.438546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439350 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.439354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.439456 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.439494 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.439595 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.439783 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":99,"thread_start_us":103,"threads_started":1}
I20260807 02:31:01.440835 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.434554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.17.53.1:45691)
0807 02:31:01.434995 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.435004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.435026 (+ 22us) server_negotiation.cc:407] Connection header received
0807 02:31:01.435130 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.435136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.435348 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.435525 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:01.435866 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.437092 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.438716 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439485 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.440311 (+ 826us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.440490 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.440541 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.440632 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":136,"threads_started":1}
I20260807 02:31:01.441231 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.433827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.17.53.2:38847)
0807 02:31:01.435281 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.435295 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.435321 (+ 26us) server_negotiation.cc:407] Connection header received
0807 02:31:01.435401 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.435407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.435591 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.435747 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:01.435993 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.436875 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.438465 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.439253 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.440609 (+ 1356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.440820 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.440879 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.441078 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1326,"thread_start_us":143,"threads_started":1}
I20260807 02:31:01.443549 12033 tablet_service.cc:1467] Processing CreateTablet for tablet 249f7d1dbd024a97a3335afb929793ae (DEFAULT_TABLE table=test-table [id=cb1fa60f8f75490caf50610f3b876a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:01.443600 11848 tablet_service.cc:1467] Processing CreateTablet for tablet 249f7d1dbd024a97a3335afb929793ae (DEFAULT_TABLE table=test-table [id=cb1fa60f8f75490caf50610f3b876a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:01.443600 12256 tablet_service.cc:1467] Processing CreateTablet for tablet 249f7d1dbd024a97a3335afb929793ae (DEFAULT_TABLE table=test-table [id=cb1fa60f8f75490caf50610f3b876a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:01.444695 11848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249f7d1dbd024a97a3335afb929793ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:01.444695 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249f7d1dbd024a97a3335afb929793ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:01.444697 12033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 249f7d1dbd024a97a3335afb929793ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:01.451344 12379 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Bootstrap starting.
I20260807 02:31:01.452841 12381 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Bootstrap starting.
I20260807 02:31:01.453094 12380 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Bootstrap starting.
I20260807 02:31:01.454139 12379 tablet_bootstrap.cc:654] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:01.455186 12381 tablet_bootstrap.cc:654] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:01.455230 12379 log.cc:824] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:01.455574 12380 tablet_bootstrap.cc:654] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:01.456071 12381 log.cc:824] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:01.456410 12380 log.cc:824] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:01.458241 12379 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: No bootstrap required, opened a new log
I20260807 02:31:01.458418 12381 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: No bootstrap required, opened a new log
I20260807 02:31:01.458496 12379 ts_tablet_manager.cc:1397] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 02:31:01.458616 12381 ts_tablet_manager.cc:1397] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:31:01.458786 12380 tablet_bootstrap.cc:492] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: No bootstrap required, opened a new log
I20260807 02:31:01.458993 12380 ts_tablet_manager.cc:1397] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 02:31:01.461634 12379 raft_consensus.cc:348] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.461901 12379 raft_consensus.cc:374] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:01.461959 12379 raft_consensus.cc:729] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e21e1fe70354cd3bdce8e0953795e39, State: Initialized, Role: FOLLOWER
I20260807 02:31:01.462509 12379 consensus_queue.cc:260] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.462966 12381 raft_consensus.cc:348] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.463029 12380 raft_consensus.cc:348] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.463227 12381 raft_consensus.cc:374] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:01.463227 12380 raft_consensus.cc:374] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:01.463296 12380 raft_consensus.cc:729] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fbc32be152046e7867e165449bc6519, State: Initialized, Role: FOLLOWER
I20260807 02:31:01.463296 12381 raft_consensus.cc:729] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c68be41519d46478dc70b2f4efe4fa4, State: Initialized, Role: FOLLOWER
I20260807 02:31:01.463482 12379 ts_tablet_manager.cc:1428] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 02:31:01.463723 12380 consensus_queue.cc:260] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.463723 12381 consensus_queue.cc:260] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.463897 12156 heartbeater.cc:503] Master 127.17.53.62:33743 was elected leader, sending a full tablet report...
I20260807 02:31:01.464391 12350 heartbeater.cc:503] Master 127.17.53.62:33743 was elected leader, sending a full tablet report...
I20260807 02:31:01.464666 12380 ts_tablet_manager.cc:1428] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 02:31:01.464666 12381 ts_tablet_manager.cc:1428] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:31:01.465219 11936 heartbeater.cc:503] Master 127.17.53.62:33743 was elected leader, sending a full tablet report...
W20260807 02:31:01.479077 11937 tablet.cc:2369] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:01.525115 12387 raft_consensus.cc:482] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:01.525367 12387 raft_consensus.cc:504] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.526647 12387 leader_election.cc:288] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e21e1fe70354cd3bdce8e0953795e39 (127.17.53.2:38847), 6c68be41519d46478dc70b2f4efe4fa4 (127.17.53.3:42287)
I20260807 02:31:01.532018 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.526976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:49207 (local address 127.17.53.2:38847)
0807 02:31:01.527180 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.527185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.527305 (+ 120us) server_negotiation.cc:407] Connection header received
0807 02:31:01.527455 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.527460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.527531 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.527614 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:01.527979 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.528801 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.530635 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.531300 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.531420 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.531487 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.531764 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.531867 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 02:31:01.532050 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.526836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:38847 (local address 127.17.53.1:49207)
0807 02:31:01.527189 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.527205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.527336 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.527641 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.527644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.527666 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:01.527815 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.527820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.528927 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.528930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.530456 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.530467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.531467 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.531471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.531587 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.531625 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.531726 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.531821 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":53,"thread_start_us":78,"threads_started":1}
I20260807 02:31:01.532159 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.527079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:52567 (local address 127.17.53.3:42287)
0807 02:31:01.527198 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.527205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.527541 (+ 336us) server_negotiation.cc:407] Connection header received
0807 02:31:01.527716 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.527720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.527779 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.527846 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:01.528163 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.528937 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.530637 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.531297 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.531420 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.531487 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.531896 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.531995 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:31:01.532307 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.526957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:42287 (local address 127.17.53.1:52567)
0807 02:31:01.527427 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.527442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:01.527547 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:01.527890 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:01.527895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:01.527918 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:01.528043 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.528048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.529047 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.529051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.530456 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:01.530467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.531584 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.531587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:01.531673 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.531729 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 02:31:01.531845 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 02:31:01.531914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":127,"thread_start_us":143,"threads_started":1}
I20260807 02:31:01.533051 12062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249f7d1dbd024a97a3335afb929793ae" candidate_uuid: "4fbc32be152046e7867e165449bc6519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e21e1fe70354cd3bdce8e0953795e39" is_pre_election: true
I20260807 02:31:01.533309 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249f7d1dbd024a97a3335afb929793ae" candidate_uuid: "4fbc32be152046e7867e165449bc6519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68be41519d46478dc70b2f4efe4fa4" is_pre_election: true
I20260807 02:31:01.533406 12062 raft_consensus.cc:2454] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fbc32be152046e7867e165449bc6519 in term 0.
I20260807 02:31:01.533622 12289 raft_consensus.cc:2454] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fbc32be152046e7867e165449bc6519 in term 0.
I20260807 02:31:01.533988 11788 leader_election.cc:302] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fbc32be152046e7867e165449bc6519, 7e21e1fe70354cd3bdce8e0953795e39; no voters:
I20260807 02:31:01.534375 12387 raft_consensus.cc:2781] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:01.534476 12387 raft_consensus.cc:482] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:01.534513 12387 raft_consensus.cc:3037] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:01.535689 12387 raft_consensus.cc:504] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.536024 12387 leader_election.cc:288] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [CANDIDATE]: Term 1 election: Requested vote from peers 7e21e1fe70354cd3bdce8e0953795e39 (127.17.53.2:38847), 6c68be41519d46478dc70b2f4efe4fa4 (127.17.53.3:42287)
I20260807 02:31:01.536412 12062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249f7d1dbd024a97a3335afb929793ae" candidate_uuid: "4fbc32be152046e7867e165449bc6519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e21e1fe70354cd3bdce8e0953795e39"
I20260807 02:31:01.536413 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "249f7d1dbd024a97a3335afb929793ae" candidate_uuid: "4fbc32be152046e7867e165449bc6519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c68be41519d46478dc70b2f4efe4fa4"
I20260807 02:31:01.536556 12289 raft_consensus.cc:3037] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:01.536556 12062 raft_consensus.cc:3037] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:01.537961 12062 raft_consensus.cc:2454] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fbc32be152046e7867e165449bc6519 in term 1.
I20260807 02:31:01.537961 12289 raft_consensus.cc:2454] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fbc32be152046e7867e165449bc6519 in term 1.
I20260807 02:31:01.538312 11788 leader_election.cc:302] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fbc32be152046e7867e165449bc6519, 7e21e1fe70354cd3bdce8e0953795e39; no voters:
I20260807 02:31:01.538544 12387 raft_consensus.cc:2781] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:01.538787 12387 raft_consensus.cc:686] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [term 1 LEADER]: Becoming Leader. State: Replica: 4fbc32be152046e7867e165449bc6519, State: Running, Role: LEADER
I20260807 02:31:01.539145 12387 consensus_queue.cc:237] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } }
I20260807 02:31:01.542303 11667 catalog_manager.cc:5495] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fbc32be152046e7867e165449bc6519 (127.17.53.1). New cstate: current_term: 1 leader_uuid: "4fbc32be152046e7867e165449bc6519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fbc32be152046e7867e165449bc6519" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 45691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 } health_report { overall_health: UNKNOWN } } }
I20260807 02:31:01.561795 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.558467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.17.53.1:45691)
0807 02:31:01.558609 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:01.558614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:01.558626 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:31:01.558755 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:01.558758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:01.558814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:01.558904 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:01.559099 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.559581 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.560403 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:01.560620 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:01.560706 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:01.561063 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:01.561463 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:01.561555 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:01.561625 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 02:31:01.561670 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:31:01.567646 12289 raft_consensus.cc:1261] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 1 FOLLOWER]: Refusing update from remote peer 4fbc32be152046e7867e165449bc6519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:01.567646 12062 raft_consensus.cc:1261] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 1 FOLLOWER]: Refusing update from remote peer 4fbc32be152046e7867e165449bc6519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:01.568406 12387 consensus_queue.cc:1035] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c68be41519d46478dc70b2f4efe4fa4" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 42287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:01.568620 12391 consensus_queue.cc:1035] T 249f7d1dbd024a97a3335afb929793ae P 4fbc32be152046e7867e165449bc6519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e21e1fe70354cd3bdce8e0953795e39" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 38847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:01.584060 11667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:01.584227 11667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:01.586854 11667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40e3a3c4cde14910ba59658ec22c23b1: Sending DeleteTablet for 3 replicas of tablet 249f7d1dbd024a97a3335afb929793ae
I20260807 02:31:01.587708 12033 tablet_service.cc:1514] Processing DeleteTablet for tablet 249f7d1dbd024a97a3335afb929793ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:01 UTC) from {username='slave'} at 127.0.0.1:40052
I20260807 02:31:01.587755 11848 tablet_service.cc:1514] Processing DeleteTablet for tablet 249f7d1dbd024a97a3335afb929793ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:01 UTC) from {username='slave'} at 127.0.0.1:53416
I20260807 02:31:01.587896 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet 249f7d1dbd024a97a3335afb929793ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:01 UTC) from {username='slave'} at 127.0.0.1:46722
I20260807 02:31:01.588373 12414 tablet_replica.cc:330] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: stopping tablet replica
I20260807 02:31:01.588370 12415 tablet_replica.cc:330] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: stopping tablet replica
I20260807 02:31:01.588698 12415 raft_consensus.cc:2229] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:01.588711 12414 raft_consensus.cc:2229] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:01.589056 12414 raft_consensus.cc:2258] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:01.589056 12415 raft_consensus.cc:2258] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:01.589116 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 11759
I20260807 02:31:01.590943 12414 ts_tablet_manager.cc:1905] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:01.590958 12415 ts_tablet_manager.cc:1905] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:01.594200 12414 ts_tablet_manager.cc:1918] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:01.594287 12414 log.cc:1196] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/249f7d1dbd024a97a3335afb929793ae
I20260807 02:31:01.594556 12414 ts_tablet_manager.cc:1939] T 249f7d1dbd024a97a3335afb929793ae P 7e21e1fe70354cd3bdce8e0953795e39: Deleting consensus metadata
I20260807 02:31:01.594947 12415 ts_tablet_manager.cc:1918] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:01.595055 12415 log.cc:1196] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/249f7d1dbd024a97a3335afb929793ae
I20260807 02:31:01.595410 12415 ts_tablet_manager.cc:1939] T 249f7d1dbd024a97a3335afb929793ae P 6c68be41519d46478dc70b2f4efe4fa4: Deleting consensus metadata
I20260807 02:31:01.595882 11645 catalog_manager.cc:4841] TS 7e21e1fe70354cd3bdce8e0953795e39 (127.17.53.2:38847): tablet 249f7d1dbd024a97a3335afb929793ae (table test-table [id=cb1fa60f8f75490caf50610f3b876a81]) successfully deleted
I20260807 02:31:01.597137 11649 catalog_manager.cc:4841] TS 6c68be41519d46478dc70b2f4efe4fa4 (127.17.53.3:42287): tablet 249f7d1dbd024a97a3335afb929793ae (table test-table [id=cb1fa60f8f75490caf50610f3b876a81]) successfully deleted
W20260807 02:31:01.603539 11642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:45691 (error 108)
I20260807 02:31:01.604439 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 11940
I20260807 02:31:01.604874 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:01.604408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:45691 (local address 127.0.0.1:53436)
0807 02:31:01.604636 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:01.604719 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 02:31:01.616662 11642 catalog_manager.cc:4584] TS 4fbc32be152046e7867e165449bc6519 (127.17.53.1:45691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 249f7d1dbd024a97a3335afb929793ae: Network error: Client connection negotiation failed: client connection to 127.17.53.1:45691: connect: Connection refused (error 111)
I20260807 02:31:01.625313 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 12160
I20260807 02:31:01.638811 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 11618
2026-08-07T02:31:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1137 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:01 dist-test-slave-g170 krb5kdc[12420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:01 dist-test-slave-g170 krb5kdc[12420](info): set up 2 sockets
Aug 07 02:31:01 dist-test-slave-g170 krb5kdc[12420](info): commencing operation
krb5kdc: starting...
W20260807 02:31:02.889832 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:02 dist-test-slave-g170 krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:03.027654 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36281
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:03.148996 12435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:03.149325 12435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:03.149396 12435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:03.153466 12435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:03.153571 12435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:03.153611 12435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:03.153649 12435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:03.153692 12435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:03.159143 12435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:36281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:03.160280 12435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:03.161381 12435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:03.170521 12449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.170502 12447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.170583 12435 server_base.cc:1034] running on GCE node
W20260807 02:31:03.170539 12452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.171620 12435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:03.172994 12435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:03.174198 12435 hybrid_clock.cc:648] HybridClock initialized: now 1786069863174183 us; error 27 us; skew 500 ppm
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:03.177101 12435 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:03.178122 12435 webserver.cc:460] Webserver started at http://127.17.53.62:34145/ using document root <none> and password file <none>
I20260807 02:31:03.178745 12435 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:03.178844 12435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:03.179086 12435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:03.181234 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "f003babfe1284c4999e8fbd880acbbe0"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "081b631bd32be39389c93094072e1a1f"
server_key_iv: "57a28d5ec0b055ca12c5d29560cf054a"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.181823 12435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "f003babfe1284c4999e8fbd880acbbe0"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "081b631bd32be39389c93094072e1a1f"
server_key_iv: "57a28d5ec0b055ca12c5d29560cf054a"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.185849 12435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:31:03.188441 12466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.189879 12435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:03.190032 12435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "f003babfe1284c4999e8fbd880acbbe0"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "081b631bd32be39389c93094072e1a1f"
server_key_iv: "57a28d5ec0b055ca12c5d29560cf054a"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.190151 12435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:03.215188 12435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:03.218874 12435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:03.219115 12435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:03.227130 12435 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:36281
I20260807 02:31:03.227165 12546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:36281 every 8 connection(s)
I20260807 02:31:03.228178 12435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:03.231310 12547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:03.235885 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 12435
I20260807 02:31:03.236052 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:03.236407 17620 external_mini_cluster.cc:1428] Setting key 22314931f901c9b9a3e31abe2d043035
I20260807 02:31:03.238034 12547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: Bootstrap starting.
I20260807 02:31:03.240715 12547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:03.241566 12547 log.cc:824] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:03.243563 12547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: No bootstrap required, opened a new log
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:03.248528 12547 raft_consensus.cc:348] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } }
I20260807 02:31:03.248867 12547 raft_consensus.cc:374] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:03.248921 12547 raft_consensus.cc:729] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003babfe1284c4999e8fbd880acbbe0, State: Initialized, Role: FOLLOWER
I20260807 02:31:03.249404 12547 consensus_queue.cc:260] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } }
I20260807 02:31:03.249591 12547 raft_consensus.cc:388] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:03.249680 12547 raft_consensus.cc:482] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:03.249800 12547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:03.251063 12547 raft_consensus.cc:504] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } }
I20260807 02:31:03.251427 12547 leader_election.cc:302] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f003babfe1284c4999e8fbd880acbbe0; no voters:
I20260807 02:31:03.251775 12547 leader_election.cc:288] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:03.251916 12553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:03.252177 12553 raft_consensus.cc:686] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [term 1 LEADER]: Becoming Leader. State: Replica: f003babfe1284c4999e8fbd880acbbe0, State: Running, Role: LEADER
I20260807 02:31:03.252512 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:03.237934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.17.53.62:36281)
0807 02:31:03.238438 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:03.238463 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:03.238501 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:31:03.239169 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:03.239187 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:03.239532 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:03.239824 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:03.240897 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.242070 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.242968 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.243457 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.243632 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:03.246589 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:03.246597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:03.246621 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:03.246659 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:03.248935 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.249509 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.249515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.249521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.249673 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.249991 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.249998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.250002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.250542 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:03.250766 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:03.251062 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 02:31:03.251393 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":202,"threads_started":1}
I20260807 02:31:03.252525 12553 consensus_queue.cc:237] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } }
I20260807 02:31:03.253181 12547 sys_catalog.cc:564] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:03.255249 12557 sys_catalog.cc:455] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f003babfe1284c4999e8fbd880acbbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } } }
I20260807 02:31:03.255676 12557 sys_catalog.cc:458] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:03.255863 12554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f003babfe1284c4999e8fbd880acbbe0. Latest consensus state: current_term: 1 leader_uuid: "f003babfe1284c4999e8fbd880acbbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f003babfe1284c4999e8fbd880acbbe0" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 36281 } } }
I20260807 02:31:03.257014 12554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0 [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:03.258550 12574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:03.258616 12574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:03.258702 12576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:03.259620 12576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:03.265515 12576 catalog_manager.cc:1347] Generated new cluster ID: 01c26e1083164765a53da5413f766687
I20260807 02:31:03.265604 12576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:03.281744 12576 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:03.282774 12576 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:03.290503 12576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: Generated new TSK 0
I20260807 02:31:03.291249 12576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:31:03.328506 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:03.461980 12586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:03.462234 12586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:03.462278 12586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:03.466497 12586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:03.466557 12586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:03.466641 12586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:31:03.471863 12586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:03.472926 12586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:03.473974 12586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:03.483741 12594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.483773 12596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.483770 12593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.483992 12586 server_base.cc:1034] running on GCE node
I20260807 02:31:03.484549 12586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:03.485208 12586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:03.486414 12586 hybrid_clock.cc:648] HybridClock initialized: now 1786069863486400 us; error 26 us; skew 500 ppm
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:03.489249 12586 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:31:03.490180 12586 webserver.cc:460] Webserver started at http://127.17.53.1:38177/ using document root <none> and password file <none>
I20260807 02:31:03.490764 12586 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:03.490878 12586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:03.491119 12586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:03.493268 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "66b22402e48349b08eb60e10db2fbfaf"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "5e04f37c8cd28bd0bc094126301ff901"
server_key_iv: "d7c2b59ff1a9029a42f35bef328d79fa"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.493858 12586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "66b22402e48349b08eb60e10db2fbfaf"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "5e04f37c8cd28bd0bc094126301ff901"
server_key_iv: "d7c2b59ff1a9029a42f35bef328d79fa"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.497929 12586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 02:31:03.500487 12606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.502090 12586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:31:03.502305 12586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "66b22402e48349b08eb60e10db2fbfaf"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "5e04f37c8cd28bd0bc094126301ff901"
server_key_iv: "d7c2b59ff1a9029a42f35bef328d79fa"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.502444 12586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:03.533341 12586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:03.536842 12586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:03.537052 12586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:03.537688 12586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:03.538580 12586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:03.538655 12586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.538722 12586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:03.538775 12586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.547969 12586 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:44907
I20260807 02:31:03.548019 12783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:44907 every 8 connection(s)
I20260807 02:31:03.548934 12586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:31:03.557188 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 12586
I20260807 02:31:03.557364 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:31:03.557758 17620 external_mini_cluster.cc:1428] Setting key 742ed956a6f8a1fa96236b0c1a35d32b
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:03.563539 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:03.550864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:60825 (local address 127.17.53.62:36281)
0807 02:31:03.551008 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:03.551012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:03.552040 (+ 1028us) server_negotiation.cc:407] Connection header received
0807 02:31:03.552972 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:03.552975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:03.553033 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:03.553121 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:03.554514 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.555129 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.556283 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.556515 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.556601 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:03.559591 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:03.559595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:03.559599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:03.559626 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:03.561359 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.561929 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.561935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.561936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.562031 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.562394 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.562398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.562400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.562596 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:03.562712 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:03.563237 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 02:31:03.563365 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:31:03.564378 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:03.551199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36281 (local address 127.17.53.1:60825)
0807 02:31:03.551889 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:03.551931 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:03.552745 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:03.553348 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:03.553355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:03.553791 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:03.554334 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:03.554369 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.555260 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.555263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:03.556160 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:03.556170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.556651 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.556654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:03.556881 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:03.557651 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:03.557673 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:03.559418 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:03.561509 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:03.561514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:03.561526 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:03.561796 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:03.562133 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:03.562137 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:03.562139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:03.562286 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:03.562756 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:03.562761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:03.562989 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 02:31:03.563181 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 02:31:03.563504 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":119,"threads_started":1}
I20260807 02:31:03.565770 12784 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36281
I20260807 02:31:03.566036 12784 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:03.566562 12784 heartbeater.cc:511] Master 127.17.53.62:36281 requested a full tablet report, sending...
I20260807 02:31:03.568296 12488 ts_manager.cc:194] Registered new tserver with Master: 66b22402e48349b08eb60e10db2fbfaf (127.17.53.1:44907)
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
I20260807 02:31:03.569705 12488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:60825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:31:03.588236 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:03.707768 12792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:03.708071 12792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:03.708119 12792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:03.712350 12792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:03.712428 12792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:03.712491 12792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:31:03.717975 12792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:03.719076 12792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:03.720105 12792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:03.730628 12800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.730721 12804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.730633 12792 server_base.cc:1034] running on GCE node
W20260807 02:31:03.730664 12801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.731356 12792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:03.732065 12792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:03.733232 12792 hybrid_clock.cc:648] HybridClock initialized: now 1786069863733215 us; error 30 us; skew 500 ppm
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:03.736541 12792 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:31:03.737762 12792 webserver.cc:460] Webserver started at http://127.17.53.2:41147/ using document root <none> and password file <none>
I20260807 02:31:03.738431 12792 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:03.738502 12792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:03.738787 12792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:03.741237 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "23356cf48a4f40b2b37389cde168a8c9"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "b94f8ed11e9e6c5b66c1d47cded66b24"
server_key_iv: "bae1968609c5e6f15807606c341d0c62"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.741881 12792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "23356cf48a4f40b2b37389cde168a8c9"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "b94f8ed11e9e6c5b66c1d47cded66b24"
server_key_iv: "bae1968609c5e6f15807606c341d0c62"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.746133 12792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 02:31:03.749104 12812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.750721 12792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:03.750934 12792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "23356cf48a4f40b2b37389cde168a8c9"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "b94f8ed11e9e6c5b66c1d47cded66b24"
server_key_iv: "bae1968609c5e6f15807606c341d0c62"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.751070 12792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:03.779124 12792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:03.782521 12792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:03.782711 12792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:03.783288 12792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:03.784127 12792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:03.784176 12792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.784217 12792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:03.784242 12792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:03.794581 12792 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:39963
I20260807 02:31:03.794603 13000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:39963 every 8 connection(s)
I20260807 02:31:03.795598 12792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:31:03.795902 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 12792
I20260807 02:31:03.796027 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:31:03.796384 17620 external_mini_cluster.cc:1428] Setting key 9365a4fb34b446714cebfe56f4fc410e
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:31:03.811362 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:03.797667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:55001 (local address 127.17.53.62:36281)
0807 02:31:03.797860 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:03.797865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:03.799211 (+ 1346us) server_negotiation.cc:407] Connection header received
0807 02:31:03.800568 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:03.800572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:03.800630 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:03.800743 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:03.802289 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.802846 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.804145 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.804375 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.804487 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:03.807203 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:03.807209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:03.807216 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:03.807240 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:03.808811 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.809360 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.809363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.809365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.809481 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:03.809904 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:03.809908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:03.809911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:03.810106 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:03.810242 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:03.810947 (+ 705us) server_negotiation.cc:299] Negotiation successful
0807 02:31:03.811101 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 02:31:03.812180 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:03.798145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36281 (local address 127.17.53.2:55001)
0807 02:31:03.799009 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:03.799073 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:03.800328 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:03.801074 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:03.801082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:03.801507 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:03.802107 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:03.802122 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.802985 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.802989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:03.803995 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:03.804005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:03.804544 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:03.804548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:03.804727 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:03.805450 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:03.805467 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:03.807025 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:03.808941 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:03.808948 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:03.808963 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:03.809258 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:03.809582 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:03.809586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:03.809588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:03.809771 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:03.810269 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:03.810280 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:03.810637 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 02:31:03.810857 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 02:31:03.811152 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":139,"threads_started":1}
I20260807 02:31:03.813620 13002 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36281
I20260807 02:31:03.813934 13002 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:03.814463 13002 heartbeater.cc:511] Master 127.17.53.62:36281 requested a full tablet report, sending...
I20260807 02:31:03.815714 12488 ts_manager.cc:194] Registered new tserver with Master: 23356cf48a4f40b2b37389cde168a8c9 (127.17.53.2:39963)
I20260807 02:31:03.816783 12488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:55001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:31:03.830166 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:03.963117 13013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:03.963438 13013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:03.963488 13013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:03.968072 13013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:03.968153 13013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:03.968226 13013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:03.974537 13013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:03.975705 13013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:03.976812 13013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:03.987495 13021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.987490 13022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:03.987490 13027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:03.987834 13013 server_base.cc:1034] running on GCE node
I20260807 02:31:03.988269 13013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:03.989097 13013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:03.990305 13013 hybrid_clock.cc:648] HybridClock initialized: now 1786069863990289 us; error 28 us; skew 500 ppm
Aug 07 02:31:03 dist-test-slave-g170 krb5kdc[12420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:03.993600 13013 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:31:03.994802 13013 webserver.cc:460] Webserver started at http://127.17.53.3:37023/ using document root <none> and password file <none>
I20260807 02:31:03.995476 13013 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:03.995584 13013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:03.995865 13013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:03.998312 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "c693e839e2704de6a847a307d1d6d345"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "70e377a07174f203072938964cbc9fb4"
server_key_iv: "6d15edb66f9062491a756706003435c3"
server_key_version: "encryptionkey@0"
I20260807 02:31:03.999001 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "c693e839e2704de6a847a307d1d6d345"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "70e377a07174f203072938964cbc9fb4"
server_key_iv: "6d15edb66f9062491a756706003435c3"
server_key_version: "encryptionkey@0"
I20260807 02:31:04.003445 13013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:04.006474 13034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:04.007951 13013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 02:31:04.008137 13013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "c693e839e2704de6a847a307d1d6d345"
format_stamp: "Formatted at 2026-08-07 02:31:03 on dist-test-slave-g170"
server_key: "70e377a07174f203072938964cbc9fb4"
server_key_iv: "6d15edb66f9062491a756706003435c3"
server_key_version: "encryptionkey@0"
I20260807 02:31:04.008278 13013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:04.030463 13013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:04.034631 13013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:04.034898 13013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:04.035640 13013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:04.036664 13013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:04.036785 13013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:04.036878 13013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:04.036940 13013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:04.047952 13013 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:36039
I20260807 02:31:04.047976 13203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:36039 every 8 connection(s)
I20260807 02:31:04.049170 13013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:04.050169 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 13013
I20260807 02:31:04.050308 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:04.050658 17620 external_mini_cluster.cc:1428] Setting key 5ac95d8a5b5ed8292d0312bc6696b59e
Aug 07 02:31:04 dist-test-slave-g170 krb5kdc[12420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069863, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:04.066058 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.051653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:42907 (local address 127.17.53.62:36281)
0807 02:31:04.051821 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.051826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.053383 (+ 1557us) server_negotiation.cc:407] Connection header received
0807 02:31:04.054580 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.054584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.054654 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.054753 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:04.056291 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.057049 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.058271 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.058577 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.058736 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.061636 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:04.061641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:04.061644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:04.061680 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:04.063406 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:04.064051 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:04.064055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:04.064056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:04.064143 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:04.064646 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:04.064651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:04.064655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:04.064914 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:04.065008 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.065738 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.065878 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:31:04.067000 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.052265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:36281 (local address 127.17.53.3:42907)
0807 02:31:04.053196 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.053265 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.054351 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.054943 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.054951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.055522 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:04.056080 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.056095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.057208 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.057211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.058116 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.058127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.058787 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.058790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.058948 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.059657 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:04.059679 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:04.061446 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:04.063559 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:04.063567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:04.063585 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:04.063904 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:04.064272 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:04.064276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:04.064278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:04.064476 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:04.065034 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:04.065044 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:04.065450 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.065681 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.066069 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":131,"threads_started":1}
I20260807 02:31:04.068353 13204 heartbeater.cc:347] Connected to a master server at 127.17.53.62:36281
I20260807 02:31:04.068704 13204 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:04.069314 13204 heartbeater.cc:511] Master 127.17.53.62:36281 requested a full tablet report, sending...
I20260807 02:31:04.070613 12488 ts_manager.cc:194] Registered new tserver with Master: c693e839e2704de6a847a307d1d6d345 (127.17.53.3:36039)
I20260807 02:31:04.071264 12488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:42907
I20260807 02:31:04.077904 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:04.088544 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.080123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.17.53.62:36281)
0807 02:31:04.080318 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.080323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.080505 (+ 182us) server_negotiation.cc:407] Connection header received
0807 02:31:04.080662 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.080670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.080713 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.080873 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:04.081607 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.082178 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.083055 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.083281 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.083385 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.085129 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:04.085134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:04.085137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:04.085167 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:04.086645 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:04.087186 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:04.087195 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:04.087196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:04.087270 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:04.087608 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:04.087611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:04.087613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:04.087835 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:04.087998 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.088279 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.088377 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 02:31:04.091806 12488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:04.095255 12488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:04.112047 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.105758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44907 (local address 127.0.0.1:51418)
0807 02:31:04.106379 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.106394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.106488 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.107866 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.107869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.107889 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:04.107969 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.107975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.109257 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.109264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.110368 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.110378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.111483 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.111486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.111592 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.111637 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.111764 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.111860 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260807 02:31:04.112124 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.106228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.17.53.1:44907)
0807 02:31:04.107388 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.107395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.107459 (+ 64us) server_negotiation.cc:407] Connection header received
0807 02:31:04.107536 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.107541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.107728 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.107871 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:04.108104 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.109142 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.110499 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.111342 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.111462 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.111633 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.111814 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.111960 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":123,"threads_started":1}
I20260807 02:31:04.112360 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.105046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39963 (local address 127.0.0.1:59384)
0807 02:31:04.105870 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.105883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.106026 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.107049 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.107055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.107078 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:04.107252 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.107260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.108847 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.108850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.110595 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.110607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.111912 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.111916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.112013 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.112037 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.112115 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.112211 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":200,"thread_start_us":122,"threads_started":1}
I20260807 02:31:04.113065 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.105047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.17.53.3:36039)
0807 02:31:04.105450 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.105465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.105962 (+ 497us) server_negotiation.cc:407] Connection header received
0807 02:31:04.106616 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.106623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.106981 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.107166 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:04.107700 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.108593 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.110769 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.111349 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.111487 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.111641 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.112835 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.112925 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":149,"threads_started":1}
I20260807 02:31:04.113318 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.105493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.17.53.2:39963)
0807 02:31:04.105878 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.105893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.105950 (+ 57us) server_negotiation.cc:407] Connection header received
0807 02:31:04.106662 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.106667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.106893 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.107068 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:04.107444 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.108682 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.110761 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.111568 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.111681 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.111826 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.112745 (+ 919us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.112881 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":113,"threads_started":1}
I20260807 02:31:04.113507 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.104858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:36039 (local address 127.0.0.1:44546)
0807 02:31:04.105810 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.105864 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.106026 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.107360 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.107362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.107379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:04.107499 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.107512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.108818 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.108822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.110595 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.110607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.112454 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.112458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.112593 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.112633 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.113246 (+ 613us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.113340 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":189,"thread_start_us":120,"threads_started":1}
I20260807 02:31:04.115619 12676 tablet_service.cc:1467] Processing CreateTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d (DEFAULT_TABLE table=test-table [id=8d8fc143accd44f6840ee1f47cbfa466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:04.116194 13110 tablet_service.cc:1467] Processing CreateTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d (DEFAULT_TABLE table=test-table [id=8d8fc143accd44f6840ee1f47cbfa466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:04.116878 12676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c18f7bda32c41c1af5bb515b8adae9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:04.117313 13110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c18f7bda32c41c1af5bb515b8adae9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:04.118659 12883 tablet_service.cc:1467] Processing CreateTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d (DEFAULT_TABLE table=test-table [id=8d8fc143accd44f6840ee1f47cbfa466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:04.119604 12883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c18f7bda32c41c1af5bb515b8adae9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:04.124892 13224 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Bootstrap starting.
I20260807 02:31:04.125384 13225 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Bootstrap starting.
I20260807 02:31:04.126621 13228 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Bootstrap starting.
I20260807 02:31:04.128877 13224 tablet_bootstrap.cc:654] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:04.129025 13228 tablet_bootstrap.cc:654] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:04.129732 13228 log.cc:824] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:04.129832 13224 log.cc:824] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:04.130033 13225 tablet_bootstrap.cc:654] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:04.131349 13225 log.cc:824] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:04.132170 13228 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: No bootstrap required, opened a new log
I20260807 02:31:04.132421 13228 ts_tablet_manager.cc:1397] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:31:04.133665 13225 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: No bootstrap required, opened a new log
I20260807 02:31:04.133876 13225 ts_tablet_manager.cc:1397] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 02:31:04.136983 13228 raft_consensus.cc:348] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.137342 13228 raft_consensus.cc:374] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:04.137413 13228 raft_consensus.cc:729] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23356cf48a4f40b2b37389cde168a8c9, State: Initialized, Role: FOLLOWER
I20260807 02:31:04.137949 13228 consensus_queue.cc:260] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.138140 13225 raft_consensus.cc:348] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.138478 13225 raft_consensus.cc:374] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:04.138500 13224 tablet_bootstrap.cc:492] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: No bootstrap required, opened a new log
I20260807 02:31:04.138538 13225 raft_consensus.cc:729] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c693e839e2704de6a847a307d1d6d345, State: Initialized, Role: FOLLOWER
I20260807 02:31:04.139050 13225 consensus_queue.cc:260] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.139240 13228 ts_tablet_manager.cc:1428] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 02:31:04.139635 13002 heartbeater.cc:503] Master 127.17.53.62:36281 was elected leader, sending a full tablet report...
I20260807 02:31:04.139973 13224 ts_tablet_manager.cc:1397] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260807 02:31:04.140139 13225 ts_tablet_manager.cc:1428] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 02:31:04.140574 13204 heartbeater.cc:503] Master 127.17.53.62:36281 was elected leader, sending a full tablet report...
I20260807 02:31:04.144922 13224 raft_consensus.cc:348] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.145200 13224 raft_consensus.cc:374] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:04.145277 13224 raft_consensus.cc:729] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b22402e48349b08eb60e10db2fbfaf, State: Initialized, Role: FOLLOWER
I20260807 02:31:04.145804 13224 consensus_queue.cc:260] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.146514 12784 heartbeater.cc:503] Master 127.17.53.62:36281 was elected leader, sending a full tablet report...
I20260807 02:31:04.146823 13224 ts_tablet_manager.cc:1428] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 02:31:04.297084 13006 tablet.cc:2369] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:31:04.300261 13205 tablet.cc:2369] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 02:31:04.300367 12785 tablet.cc:2369] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:04.509135 13233 raft_consensus.cc:482] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:04.509387 13233 raft_consensus.cc:504] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.510576 13233 leader_election.cc:288] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66b22402e48349b08eb60e10db2fbfaf (127.17.53.1:44907), 23356cf48a4f40b2b37389cde168a8c9 (127.17.53.2:39963)
I20260807 02:31:04.515107 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.511219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:40257 (local address 127.17.53.2:39963)
0807 02:31:04.511363 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.511369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.511386 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:31:04.511449 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.511454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.511522 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.511607 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:04.511957 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.512623 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514063 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.514473 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514555 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.514612 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.514879 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.514969 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:31:04.515122 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.510933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:39963 (local address 127.17.53.3:40257)
0807 02:31:04.511059 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.511077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.511207 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.511645 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.511649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.511679 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:04.511829 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.511840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.512747 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.512751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.513928 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.513939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.514602 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.514734 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.514764 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.514851 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.514941 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 02:31:04.515202 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.510933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44907 (local address 127.17.53.3:41381)
0807 02:31:04.511422 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.511437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:04.511567 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:04.511893 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:04.511915 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:04.511938 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:04.512072 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.512079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.513097 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.513100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.513972 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:04.513982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.514759 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:04.514882 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.514909 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:31:04.514985 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 02:31:04.515053 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":46,"thread_start_us":100,"threads_started":1}
I20260807 02:31:04.515275 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.511071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:41381 (local address 127.17.53.1:44907)
0807 02:31:04.511205 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.511209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.511563 (+ 354us) server_negotiation.cc:407] Connection header received
0807 02:31:04.511708 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.511712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.511774 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.511879 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:04.512247 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.512978 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514140 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.514630 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.514748 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.514827 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.515029 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.515134 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:31:04.515940 12921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c18f7bda32c41c1af5bb515b8adae9d" candidate_uuid: "c693e839e2704de6a847a307d1d6d345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23356cf48a4f40b2b37389cde168a8c9" is_pre_election: true
I20260807 02:31:04.516101 12711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c18f7bda32c41c1af5bb515b8adae9d" candidate_uuid: "c693e839e2704de6a847a307d1d6d345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b22402e48349b08eb60e10db2fbfaf" is_pre_election: true
I20260807 02:31:04.516326 12921 raft_consensus.cc:2454] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c693e839e2704de6a847a307d1d6d345 in term 0.
I20260807 02:31:04.516402 12711 raft_consensus.cc:2454] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c693e839e2704de6a847a307d1d6d345 in term 0.
I20260807 02:31:04.516957 13038 leader_election.cc:302] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23356cf48a4f40b2b37389cde168a8c9, 66b22402e48349b08eb60e10db2fbfaf, c693e839e2704de6a847a307d1d6d345; no voters:
I20260807 02:31:04.517274 13233 raft_consensus.cc:2781] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:04.517367 13233 raft_consensus.cc:482] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:04.517411 13233 raft_consensus.cc:3037] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:04.518479 13233 raft_consensus.cc:504] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.518859 13233 leader_election.cc:288] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [CANDIDATE]: Term 1 election: Requested vote from peers 66b22402e48349b08eb60e10db2fbfaf (127.17.53.1:44907), 23356cf48a4f40b2b37389cde168a8c9 (127.17.53.2:39963)
I20260807 02:31:04.519234 12711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c18f7bda32c41c1af5bb515b8adae9d" candidate_uuid: "c693e839e2704de6a847a307d1d6d345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b22402e48349b08eb60e10db2fbfaf"
I20260807 02:31:04.519371 12711 raft_consensus.cc:3037] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:04.519443 12921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c18f7bda32c41c1af5bb515b8adae9d" candidate_uuid: "c693e839e2704de6a847a307d1d6d345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23356cf48a4f40b2b37389cde168a8c9"
I20260807 02:31:04.519618 12921 raft_consensus.cc:3037] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:04.520519 12711 raft_consensus.cc:2454] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c693e839e2704de6a847a307d1d6d345 in term 1.
I20260807 02:31:04.520875 13038 leader_election.cc:302] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b22402e48349b08eb60e10db2fbfaf, c693e839e2704de6a847a307d1d6d345; no voters:
I20260807 02:31:04.520905 12921 raft_consensus.cc:2454] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c693e839e2704de6a847a307d1d6d345 in term 1.
I20260807 02:31:04.521064 13233 raft_consensus.cc:2781] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:04.521319 13233 raft_consensus.cc:686] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 1 LEADER]: Becoming Leader. State: Replica: c693e839e2704de6a847a307d1d6d345, State: Running, Role: LEADER
I20260807 02:31:04.521730 13233 consensus_queue.cc:237] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } }
I20260807 02:31:04.525405 12488 catalog_manager.cc:5495] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 reported cstate change: term changed from 0 to 1, leader changed from <none> to c693e839e2704de6a847a307d1d6d345 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "c693e839e2704de6a847a307d1d6d345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c693e839e2704de6a847a307d1d6d345" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 36039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 } health_report { overall_health: UNKNOWN } } }
I20260807 02:31:04.571424 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.567654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.17.53.3:36039)
0807 02:31:04.567856 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.567862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.567876 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:31:04.567957 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.567959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.568014 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.568141 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:04.568433 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.568974 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.569862 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.570082 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.570200 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.570540 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:04.570953 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:04.571126 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.571214 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.571256 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 02:31:04.577790 12711 raft_consensus.cc:1261] T 2c18f7bda32c41c1af5bb515b8adae9d P 66b22402e48349b08eb60e10db2fbfaf [term 1 FOLLOWER]: Refusing update from remote peer c693e839e2704de6a847a307d1d6d345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:04.577790 12921 raft_consensus.cc:1261] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 1 FOLLOWER]: Refusing update from remote peer c693e839e2704de6a847a307d1d6d345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:04.578557 13233 consensus_queue.cc:1035] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b22402e48349b08eb60e10db2fbfaf" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 44907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:04.578785 13237 consensus_queue.cc:1035] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23356cf48a4f40b2b37389cde168a8c9" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 39963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:04.588093 13242 mvcc.cc:205] Tried to move back new op lower bound from 7315742165302181888 to 7315742165083627520. Current Snapshot: MvccSnapshot[applied={T|T < 7315742165302181888}]
I20260807 02:31:04.589985 13241 mvcc.cc:205] Tried to move back new op lower bound from 7315742165302181888 to 7315742165083627520. Current Snapshot: MvccSnapshot[applied={T|T < 7315742165302181888}]
I20260807 02:31:04.596127 12487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:04.596285 12487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:04.599035 12487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f003babfe1284c4999e8fbd880acbbe0: Sending DeleteTablet for 3 replicas of tablet 2c18f7bda32c41c1af5bb515b8adae9d
I20260807 02:31:04.600122 12883 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:04 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:59384
I20260807 02:31:04.600232 13110 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:04 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:44546
I20260807 02:31:04.600667 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 12586
I20260807 02:31:04.600699 12676 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c18f7bda32c41c1af5bb515b8adae9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:04 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:51418
I20260807 02:31:04.606967 13257 tablet_replica.cc:330] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: stopping tablet replica
I20260807 02:31:04.607445 13257 raft_consensus.cc:2229] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:31:04.608003 13257 raft_consensus.cc:2258] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:04.609009 13255 tablet_replica.cc:330] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: stopping tablet replica
I20260807 02:31:04.609441 13255 raft_consensus.cc:2229] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:04.609889 13255 raft_consensus.cc:2258] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:04.611200 13257 ts_tablet_manager.cc:1905] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:04.611563 13255 ts_tablet_manager.cc:1905] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:04.614634 13255 ts_tablet_manager.cc:1918] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:04.614832 13255 log.cc:1196] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/2c18f7bda32c41c1af5bb515b8adae9d
I20260807 02:31:04.614989 13257 ts_tablet_manager.cc:1918] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:04.615105 13257 log.cc:1196] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/2c18f7bda32c41c1af5bb515b8adae9d
I20260807 02:31:04.615383 13255 ts_tablet_manager.cc:1939] T 2c18f7bda32c41c1af5bb515b8adae9d P 23356cf48a4f40b2b37389cde168a8c9: Deleting consensus metadata
I20260807 02:31:04.615438 13257 ts_tablet_manager.cc:1939] T 2c18f7bda32c41c1af5bb515b8adae9d P c693e839e2704de6a847a307d1d6d345: Deleting consensus metadata
I20260807 02:31:04.616909 12468 catalog_manager.cc:4841] TS 23356cf48a4f40b2b37389cde168a8c9 (127.17.53.2:39963): tablet 2c18f7bda32c41c1af5bb515b8adae9d (table test-table [id=8d8fc143accd44f6840ee1f47cbfa466]) successfully deleted
I20260807 02:31:04.617172 12472 catalog_manager.cc:4841] TS c693e839e2704de6a847a307d1d6d345 (127.17.53.3:36039): tablet 2c18f7bda32c41c1af5bb515b8adae9d (table test-table [id=8d8fc143accd44f6840ee1f47cbfa466]) successfully deleted
W20260807 02:31:04.619716 12472 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:44907 (error 108)
I20260807 02:31:04.619923 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 12792
I20260807 02:31:04.627663 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.627227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:44907 (local address 127.0.0.1:51428)
0807 02:31:04.627492 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:04.627567 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
W20260807 02:31:04.627952 12472 catalog_manager.cc:4584] TS 66b22402e48349b08eb60e10db2fbfaf (127.17.53.1:44907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c18f7bda32c41c1af5bb515b8adae9d: Network error: Client connection negotiation failed: client connection to 127.17.53.1:44907: connect: Connection refused (error 111)
I20260807 02:31:04.635485 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 13013
I20260807 02:31:04.657671 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 12435
2026-08-07T02:31:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3023 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T02:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:04Z Disabled control of system clock
I20260807 02:31:04.699609 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41773
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:04.820899 13271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:04.821197 13271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:04.821235 13271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:04.825363 13271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:04.825439 13271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:04.825455 13271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:04.825472 13271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:04.825523 13271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:04.830439 13271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:04.831570 13271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:04.833031 13271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:04.842168 13277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:04.842159 13276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:04.842417 13271 server_base.cc:1034] running on GCE node
W20260807 02:31:04.842172 13286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:04.843207 13271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:04.844450 13271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:04.845649 13271 hybrid_clock.cc:648] HybridClock initialized: now 1786069864845617 us; error 59 us; skew 500 ppm
I20260807 02:31:04.853205 13271 webserver.cc:460] Webserver started at http://127.17.53.62:38131/ using document root <none> and password file <none>
I20260807 02:31:04.853914 13271 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:04.854027 13271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:04.854305 13271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:04.857003 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "16806a811669448dad6382a6537bc319"
format_stamp: "Formatted at 2026-08-07 02:31:04 on dist-test-slave-g170"
server_key: "a13b2ea71c695f5d92d465911ccc5b9b"
server_key_iv: "9f578bb11bc9984cdb89ecedde6c3bea"
server_key_version: "encryptionkey@0"
I20260807 02:31:04.857842 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "16806a811669448dad6382a6537bc319"
format_stamp: "Formatted at 2026-08-07 02:31:04 on dist-test-slave-g170"
server_key: "a13b2ea71c695f5d92d465911ccc5b9b"
server_key_iv: "9f578bb11bc9984cdb89ecedde6c3bea"
server_key_version: "encryptionkey@0"
I20260807 02:31:04.863826 13271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 02:31:04.867672 13292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:04.869333 13271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:04.869525 13271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "16806a811669448dad6382a6537bc319"
format_stamp: "Formatted at 2026-08-07 02:31:04 on dist-test-slave-g170"
server_key: "a13b2ea71c695f5d92d465911ccc5b9b"
server_key_iv: "9f578bb11bc9984cdb89ecedde6c3bea"
server_key_version: "encryptionkey@0"
I20260807 02:31:04.869692 13271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:04.917795 13271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:04.918613 13271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:04.918834 13271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:04.926795 13271 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:41773
I20260807 02:31:04.926892 13364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:41773 every 8 connection(s)
I20260807 02:31:04.927857 13271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:04.929657 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 13271
I20260807 02:31:04.929867 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:04.930236 17620 external_mini_cluster.cc:1428] Setting key 8b11048d36437577b8fe4fbb36e671b1
I20260807 02:31:04.932183 13365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:04.938686 13365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: Bootstrap starting.
I20260807 02:31:04.940719 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:04.931852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.17.53.62:41773)
0807 02:31:04.932673 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:04.932692 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:04.932737 (+ 45us) server_negotiation.cc:407] Connection header received
0807 02:31:04.933555 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:04.933574 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:04.934024 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:04.934435 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:04.934919 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.936007 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.937057 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:04.937594 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:04.937793 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:04.938839 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:04.938846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:04.938871 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:04.939307 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:04.939366 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:04.939388 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:04.939512 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:04.939646 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 02:31:04.939938 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":159,"threads_started":1}
I20260807 02:31:04.941427 13365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:04.942281 13365 log.cc:824] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:04.944535 13365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: No bootstrap required, opened a new log
I20260807 02:31:04.948298 13365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } }
I20260807 02:31:04.948571 13365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:04.948659 13365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16806a811669448dad6382a6537bc319, State: Initialized, Role: FOLLOWER
I20260807 02:31:04.949252 13365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } }
I20260807 02:31:04.949424 13365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:04.949496 13365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:04.949600 13365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:04.950598 13365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } }
I20260807 02:31:04.950964 13365 leader_election.cc:302] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16806a811669448dad6382a6537bc319; no voters:
I20260807 02:31:04.951313 13365 leader_election.cc:288] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:04.951507 13375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:04.951814 13375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [term 1 LEADER]: Becoming Leader. State: Replica: 16806a811669448dad6382a6537bc319, State: Running, Role: LEADER
I20260807 02:31:04.952138 13375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } }
I20260807 02:31:04.952737 13365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:04.953941 13377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16806a811669448dad6382a6537bc319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } } }
I20260807 02:31:04.954093 13377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:04.954442 13379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16806a811669448dad6382a6537bc319. Latest consensus state: current_term: 1 leader_uuid: "16806a811669448dad6382a6537bc319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16806a811669448dad6382a6537bc319" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41773 } } }
I20260807 02:31:04.954547 13379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:04.954638 13388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:04.958110 13388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:04.964558 13388 catalog_manager.cc:1347] Generated new cluster ID: 8dbae0a376f14cc686ac6ce4385cb4d8
I20260807 02:31:04.964660 13388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:04.986498 13388 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:04.987644 13388 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:04.999054 13388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: Generated new TSK 0
I20260807 02:31:04.999927 13388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:05.011523 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--builtin_ntp_servers=127.17.53.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:05.152493 13401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:05.152828 13401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:05.152941 13401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:05.157241 13401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:05.157320 13401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:05.157413 13401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:31:05.162384 13401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:05.163561 13401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:05.165067 13401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:05.175143 13412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.175225 13406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.175115 13407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:05.175518 13401 server_base.cc:1034] running on GCE node
I20260807 02:31:05.175933 13401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:05.176587 13401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:05.177894 13401 hybrid_clock.cc:648] HybridClock initialized: now 1786069865177880 us; error 37 us; skew 500 ppm
I20260807 02:31:05.180085 13401 webserver.cc:460] Webserver started at http://127.17.53.1:46391/ using document root <none> and password file <none>
I20260807 02:31:05.180807 13401 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:05.180927 13401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:05.181195 13401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:05.183385 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "ccf2556364584a80a09397879eda2174"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "feafd50f4dbb3fce2b072c564792e3b9"
server_key_iv: "016857f2c2a443a510dc76e104541b14"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.184023 13401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "ccf2556364584a80a09397879eda2174"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "feafd50f4dbb3fce2b072c564792e3b9"
server_key_iv: "016857f2c2a443a510dc76e104541b14"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.187996 13401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:05.190989 13420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.192322 13401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:31:05.192422 13401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "ccf2556364584a80a09397879eda2174"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "feafd50f4dbb3fce2b072c564792e3b9"
server_key_iv: "016857f2c2a443a510dc76e104541b14"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.192509 13401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:05.206740 13401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:05.207369 13401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:05.207522 13401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:05.208067 13401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:05.209055 13401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:05.209106 13401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.209148 13401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:05.209170 13401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.218930 13401 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:33579
I20260807 02:31:05.218951 13594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:33579 every 8 connection(s)
I20260807 02:31:05.219935 13401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:31:05.221414 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 13401
I20260807 02:31:05.221553 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:31:05.221966 17620 external_mini_cluster.cc:1428] Setting key d485ff25679115e4012d067c6db8c993
I20260807 02:31:05.224324 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--builtin_ntp_servers=127.17.53.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:05.231432 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.222086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:43519 (local address 127.17.53.62:41773)
0807 02:31:05.222242 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.222247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.223776 (+ 1529us) server_negotiation.cc:407] Connection header received
0807 02:31:05.225346 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.225352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.225425 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.225527 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:05.226689 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.227223 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.229067 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.229313 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.229422 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.230828 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:05.230831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:05.230834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:05.230847 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:05.230869 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:05.230877 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:05.230943 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.231221 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.231304 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:31:05.232786 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.222633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41773 (local address 127.17.53.1:43519)
0807 02:31:05.223515 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.223573 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.225117 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.225697 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.225704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.225796 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:05.226520 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.226543 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.227367 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.227372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.228927 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.228943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.229505 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.229511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.229750 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.230461 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:05.230465 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:05.230531 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:05.230628 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:05.230979 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:05.230992 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:05.231001 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.231159 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.231543 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":128,"threads_started":1}
I20260807 02:31:05.234414 13597 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41773
I20260807 02:31:05.234714 13597 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:05.235482 13597 heartbeater.cc:511] Master 127.17.53.62:41773 requested a full tablet report, sending...
I20260807 02:31:05.237340 13313 ts_manager.cc:194] Registered new tserver with Master: ccf2556364584a80a09397879eda2174 (127.17.53.1:33579)
I20260807 02:31:05.238564 13313 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.1:43519
W20260807 02:31:05.346824 13600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:05.347132 13600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:05.347208 13600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:05.351297 13600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:05.351375 13600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:05.351462 13600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:31:05.357393 13600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:05.358526 13600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:05.359932 13600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:05.369978 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.369967 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.369967 13605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:05.370051 13600 server_base.cc:1034] running on GCE node
I20260807 02:31:05.370642 13600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:05.371325 13600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:05.372514 13600 hybrid_clock.cc:648] HybridClock initialized: now 1786069865372486 us; error 43 us; skew 500 ppm
I20260807 02:31:05.374619 13600 webserver.cc:460] Webserver started at http://127.17.53.2:32781/ using document root <none> and password file <none>
I20260807 02:31:05.375315 13600 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:05.375415 13600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:05.375651 13600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:05.377789 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "8ba4dd7ef6674719ab49eaf7e9936230"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "bf0e9fb4f0c4d3fa371f277dbd7dbf10"
server_key_iv: "8580bd3e26c0d40f617d11b4a8f68545"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.378389 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "8ba4dd7ef6674719ab49eaf7e9936230"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "bf0e9fb4f0c4d3fa371f277dbd7dbf10"
server_key_iv: "8580bd3e26c0d40f617d11b4a8f68545"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.382138 13600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:31:05.384580 13621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.385910 13600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:31:05.386055 13600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "8ba4dd7ef6674719ab49eaf7e9936230"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "bf0e9fb4f0c4d3fa371f277dbd7dbf10"
server_key_iv: "8580bd3e26c0d40f617d11b4a8f68545"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.386171 13600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:05.422758 13600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:05.423563 13600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:05.423776 13600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:05.424445 13600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:05.425513 13600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:05.425595 13600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.425666 13600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:05.425712 13600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.435868 13600 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:36603
I20260807 02:31:05.435894 13797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:36603 every 8 connection(s)
I20260807 02:31:05.436923 13600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:31:05.443552 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 13600
I20260807 02:31:05.443704 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:31:05.443984 17620 external_mini_cluster.cc:1428] Setting key 9524b59edaeef9d01d350d579757953a
I20260807 02:31:05.446265 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--builtin_ntp_servers=127.17.53.20:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:05.447959 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.439167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:58259 (local address 127.17.53.62:41773)
0807 02:31:05.439346 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.439351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.440613 (+ 1262us) server_negotiation.cc:407] Connection header received
0807 02:31:05.442161 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.442164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.442216 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.442319 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:05.443544 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.444221 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.445716 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.445934 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.446027 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.447412 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:05.447415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:05.447417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:05.447430 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:05.447450 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:05.447457 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:05.447527 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.447751 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.447843 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 02:31:05.449044 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.439660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41773 (local address 127.17.53.2:58259)
0807 02:31:05.440439 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.440488 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.441921 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.442512 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.442519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.442605 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:05.443356 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.443382 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.444380 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.444384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.445589 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.445602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.446110 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.446117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.446350 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.447060 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:05.447065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:05.447126 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:05.447230 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:05.447554 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:05.447567 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:05.447576 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.447715 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.448047 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":177,"threads_started":1}
I20260807 02:31:05.450315 13799 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41773
I20260807 02:31:05.450583 13799 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:05.451129 13799 heartbeater.cc:511] Master 127.17.53.62:41773 requested a full tablet report, sending...
I20260807 02:31:05.452288 13313 ts_manager.cc:194] Registered new tserver with Master: 8ba4dd7ef6674719ab49eaf7e9936230 (127.17.53.2:36603)
I20260807 02:31:05.452975 13313 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.2:58259
W20260807 02:31:05.570886 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:05.571142 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:05.571192 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:05.579537 13802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:05.579620 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:05.579712 13802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:05.586001 13802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:41773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:05.587172 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:05.588645 13802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:05.599778 13814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.599740 13809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:05.599740 13811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:05.600011 13802 server_base.cc:1034] running on GCE node
I20260807 02:31:05.600613 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:05.601388 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:05.602574 13802 hybrid_clock.cc:648] HybridClock initialized: now 1786069865602553 us; error 32 us; skew 500 ppm
I20260807 02:31:05.604585 13802 webserver.cc:460] Webserver started at http://127.17.53.3:42901/ using document root <none> and password file <none>
I20260807 02:31:05.605275 13802 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:05.605342 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:05.605569 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:05.607685 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "af848c72db4982a50f82c5a72e135dbb"
server_key_iv: "a17278a60a2c3d64b3cb9033c1dcca29"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.608273 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "af848c72db4982a50f82c5a72e135dbb"
server_key_iv: "a17278a60a2c3d64b3cb9033c1dcca29"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.612198 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:05.614992 13823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.616376 13802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:31:05.616521 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3"
format_stamp: "Formatted at 2026-08-07 02:31:05 on dist-test-slave-g170"
server_key: "af848c72db4982a50f82c5a72e135dbb"
server_key_iv: "a17278a60a2c3d64b3cb9033c1dcca29"
server_key_version: "encryptionkey@0"
I20260807 02:31:05.616647 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:05.635205 13802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:05.635968 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:05.636193 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:05.636924 13802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:05.637948 13802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:05.638000 13802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.638041 13802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:05.638104 13802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:05.648422 13802 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:44985
I20260807 02:31:05.648444 13994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:44985 every 8 connection(s)
I20260807 02:31:05.649556 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:05.655462 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 13802
I20260807 02:31:05.655611 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:05.655965 17620 external_mini_cluster.cc:1428] Setting key 85aea658f163a88f25a8ef8d04397791
I20260807 02:31:05.659988 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.651583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:36365 (local address 127.17.53.62:41773)
0807 02:31:05.651805 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.651811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.653050 (+ 1239us) server_negotiation.cc:407] Connection header received
0807 02:31:05.654564 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.654567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.654670 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.654770 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:05.656103 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.656620 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.657913 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.658234 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.658353 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.659479 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:05.659483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:05.659486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:05.659498 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:05.659529 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:05.659537 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:05.659619 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.659785 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.659845 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 02:31:05.660943 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.652043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41773 (local address 127.17.53.3:36365)
0807 02:31:05.652889 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.652933 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.654365 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.654972 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.654980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.655063 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:05.655892 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.655914 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.656848 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.656853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.657754 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.657763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.658407 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.658412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.658574 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.659205 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:05.659207 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:05.659245 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:05.659308 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:05.659622 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:05.659634 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:05.659643 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.659782 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.660060 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":135,"threads_started":1}
I20260807 02:31:05.662025 13995 heartbeater.cc:347] Connected to a master server at 127.17.53.62:41773
I20260807 02:31:05.662283 13995 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:05.662942 13995 heartbeater.cc:511] Master 127.17.53.62:41773 requested a full tablet report, sending...
I20260807 02:31:05.664017 13313 ts_manager.cc:194] Registered new tserver with Master: 9bb1d13bf4cf46d58c25a3ef1787acf3 (127.17.53.3:44985)
I20260807 02:31:05.664604 13313 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.3:36365
I20260807 02:31:05.670136 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:05.676084 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.672262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57844 (local address 127.17.53.62:41773)
0807 02:31:05.672393 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.672397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.672481 (+ 84us) server_negotiation.cc:407] Connection header received
0807 02:31:05.672583 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.672585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.672621 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.672694 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:05.672944 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.673434 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.674342 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.674592 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.674666 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.675635 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:05.675638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:05.675641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:05.675655 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:05.675677 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:05.675687 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:05.675816 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.675926 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.675961 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:31:05.679471 13313 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:05.682516 13313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:05.697969 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.691430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33579 (local address 127.0.0.1:37478)
0807 02:31:05.691956 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.692004 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.692138 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.693823 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.693827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.693846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:05.693941 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.693947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.695251 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.695254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.696493 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.696501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697424 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.697533 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697570 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.697682 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.697803 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":91,"thread_start_us":71,"threads_started":1}
I20260807 02:31:05.697973 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.691579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.17.53.1:33579)
0807 02:31:05.693175 (+ 1596us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.693183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.693211 (+ 28us) server_negotiation.cc:407] Connection header received
0807 02:31:05.693294 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.693300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.693499 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.693649 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:05.694074 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.695138 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.696639 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697300 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697422 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697621 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.697709 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.697806 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1494,"thread_start_us":137,"threads_started":1}
I20260807 02:31:05.698309 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.692511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.17.53.3:44985)
0807 02:31:05.692992 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.693004 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.693287 (+ 283us) server_negotiation.cc:407] Connection header received
0807 02:31:05.693402 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.693410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.693588 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.693763 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:05.693958 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.694691 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.696487 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697343 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697450 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697642 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.698030 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.698159 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":134,"threads_started":1}
I20260807 02:31:05.698725 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.691633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:36603 (local address 127.0.0.1:48412)
0807 02:31:05.692880 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.692895 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.693002 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.693459 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.693464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.693484 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:05.693636 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.693644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.695100 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.695104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.696489 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.696501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697760 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.697835 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697858 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.697920 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.697997 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"mutex_wait_us":76,"thread_start_us":95,"threads_started":1}
I20260807 02:31:05.699167 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.692083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:44985 (local address 127.0.0.1:35180)
0807 02:31:05.693169 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.693182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.693294 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.693740 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.693743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.693763 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:05.693860 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.693869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.694964 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.694968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.696355 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.696367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697711 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.697804 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697828 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.697902 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.699047 (+ 1145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":1060,"thread_start_us":94,"threads_started":1}
I20260807 02:31:05.699594 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.692232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.17.53.2:36603)
0807 02:31:05.692701 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.692714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.693013 (+ 299us) server_negotiation.cc:407] Connection header received
0807 02:31:05.693106 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.693109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.693313 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.693479 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:05.693772 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.694967 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.696660 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.697447 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.697541 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.697709 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.699337 (+ 1628us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.699435 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":155,"threads_started":1}
I20260807 02:31:05.701376 13498 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 (DEFAULT_TABLE table=test-table [id=21fb7eabd8d044c8a222aa4d9790c5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:05.702134 13694 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 (DEFAULT_TABLE table=test-table [id=21fb7eabd8d044c8a222aa4d9790c5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:05.702023 13895 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 (DEFAULT_TABLE table=test-table [id=21fb7eabd8d044c8a222aa4d9790c5e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:05.702621 13498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfd98d91597433bbfd625ddce4ca6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:05.703059 13694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfd98d91597433bbfd625ddce4ca6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:05.703094 13895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfd98d91597433bbfd625ddce4ca6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:05.709586 14018 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Bootstrap starting.
I20260807 02:31:05.710799 14021 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: Bootstrap starting.
I20260807 02:31:05.711755 14018 tablet_bootstrap.cc:654] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:05.712527 14018 log.cc:824] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:05.712567 14021 tablet_bootstrap.cc:654] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:05.713160 14020 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Bootstrap starting.
I20260807 02:31:05.713512 14021 log.cc:824] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:05.714788 14018 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: No bootstrap required, opened a new log
I20260807 02:31:05.714993 14018 ts_tablet_manager.cc:1397] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:31:05.715307 14020 tablet_bootstrap.cc:654] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:05.715655 14021 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: No bootstrap required, opened a new log
I20260807 02:31:05.715843 14021 ts_tablet_manager.cc:1397] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 02:31:05.716107 14020 log.cc:824] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:05.717742 14020 tablet_bootstrap.cc:492] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: No bootstrap required, opened a new log
I20260807 02:31:05.717944 14020 ts_tablet_manager.cc:1397] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 02:31:05.720253 14021 raft_consensus.cc:348] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.720566 14021 raft_consensus.cc:374] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:05.720625 14021 raft_consensus.cc:729] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf2556364584a80a09397879eda2174, State: Initialized, Role: FOLLOWER
I20260807 02:31:05.720686 14018 raft_consensus.cc:348] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.720957 14018 raft_consensus.cc:374] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:05.721014 14018 raft_consensus.cc:729] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba4dd7ef6674719ab49eaf7e9936230, State: Initialized, Role: FOLLOWER
I20260807 02:31:05.721130 14021 consensus_queue.cc:260] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.721438 14018 consensus_queue.cc:260] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.721969 13597 heartbeater.cc:503] Master 127.17.53.62:41773 was elected leader, sending a full tablet report...
I20260807 02:31:05.722245 14021 ts_tablet_manager.cc:1428] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 02:31:05.722295 14020 raft_consensus.cc:348] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.722555 14020 raft_consensus.cc:374] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:05.722616 14020 raft_consensus.cc:729] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb1d13bf4cf46d58c25a3ef1787acf3, State: Initialized, Role: FOLLOWER
I20260807 02:31:05.722940 14018 ts_tablet_manager.cc:1428] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 02:31:05.723039 14020 consensus_queue.cc:260] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.723735 13995 heartbeater.cc:503] Master 127.17.53.62:41773 was elected leader, sending a full tablet report...
I20260807 02:31:05.723968 13799 heartbeater.cc:503] Master 127.17.53.62:41773 was elected leader, sending a full tablet report...
I20260807 02:31:05.724009 14020 ts_tablet_manager.cc:1428] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:31:05.728420 14029 raft_consensus.cc:482] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:05.728547 14029 raft_consensus.cc:504] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.729537 14029 leader_election.cc:288] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccf2556364584a80a09397879eda2174 (127.17.53.1:33579), 9bb1d13bf4cf46d58c25a3ef1787acf3 (127.17.53.3:44985)
I20260807 02:31:05.733536 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.729815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33579 (local address 127.17.53.2:55037)
0807 02:31:05.730029 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.730047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.730138 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.730433 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.730436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.730455 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:05.730679 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.730694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.731506 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.731509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.732341 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.732351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.733057 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.733060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.733191 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.733234 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.733309 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.733423 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":90}
I20260807 02:31:05.733541 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.729899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60861 (local address 127.17.53.3:44985)
0807 02:31:05.730310 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.730314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.730326 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:31:05.730432 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.730435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.730476 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.730545 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:05.730879 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.731353 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.732472 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.732928 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.733047 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.733104 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.733338 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.733414 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321}
I20260807 02:31:05.733563 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.729866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:44985 (local address 127.17.53.2:60861)
0807 02:31:05.730208 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.730223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:05.730315 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:05.730574 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:05.730576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:05.730591 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:05.730753 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.730769 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.731463 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.731466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.732342 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:05.732352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.733051 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.733054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:05.733206 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.733225 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 02:31:05.733317 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 02:31:05.733415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":84,"thread_start_us":125,"threads_started":1}
I20260807 02:31:05.733944 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.729965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:55037 (local address 127.17.53.1:33579)
0807 02:31:05.730077 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.730082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.730098 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:31:05.730252 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.730259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.730325 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.730403 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:05.730803 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.731365 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.732471 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.732929 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.733041 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.733101 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.733743 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.733832 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 02:31:05.734395 13931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfd98d91597433bbfd625ddce4ca6f8" candidate_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" is_pre_election: true
I20260807 02:31:05.734719 13931 raft_consensus.cc:2454] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ba4dd7ef6674719ab49eaf7e9936230 in term 0.
I20260807 02:31:05.734751 13526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfd98d91597433bbfd625ddce4ca6f8" candidate_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf2556364584a80a09397879eda2174" is_pre_election: true
I20260807 02:31:05.735055 13526 raft_consensus.cc:2454] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ba4dd7ef6674719ab49eaf7e9936230 in term 0.
I20260807 02:31:05.735251 13624 leader_election.cc:302] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba4dd7ef6674719ab49eaf7e9936230, 9bb1d13bf4cf46d58c25a3ef1787acf3; no voters:
I20260807 02:31:05.735575 14029 raft_consensus.cc:2781] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:05.735682 14029 raft_consensus.cc:482] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:05.735739 14029 raft_consensus.cc:3037] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:05.736901 14029 raft_consensus.cc:504] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.737293 14029 leader_election.cc:288] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [CANDIDATE]: Term 1 election: Requested vote from peers ccf2556364584a80a09397879eda2174 (127.17.53.1:33579), 9bb1d13bf4cf46d58c25a3ef1787acf3 (127.17.53.3:44985)
I20260807 02:31:05.737677 13931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfd98d91597433bbfd625ddce4ca6f8" candidate_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3"
I20260807 02:31:05.737807 13526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfd98d91597433bbfd625ddce4ca6f8" candidate_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccf2556364584a80a09397879eda2174"
I20260807 02:31:05.737843 13931 raft_consensus.cc:3037] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:05.737982 13526 raft_consensus.cc:3037] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:05.739079 13526 raft_consensus.cc:2454] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba4dd7ef6674719ab49eaf7e9936230 in term 1.
I20260807 02:31:05.739084 13931 raft_consensus.cc:2454] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba4dd7ef6674719ab49eaf7e9936230 in term 1.
I20260807 02:31:05.739506 13623 leader_election.cc:302] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba4dd7ef6674719ab49eaf7e9936230, ccf2556364584a80a09397879eda2174; no voters:
I20260807 02:31:05.739739 14029 raft_consensus.cc:2781] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:05.739985 14029 raft_consensus.cc:686] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 1 LEADER]: Becoming Leader. State: Replica: 8ba4dd7ef6674719ab49eaf7e9936230, State: Running, Role: LEADER
I20260807 02:31:05.740362 14029 consensus_queue.cc:237] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } }
I20260807 02:31:05.744278 13312 catalog_manager.cc:5495] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ba4dd7ef6674719ab49eaf7e9936230 (127.17.53.2). New cstate: current_term: 1 leader_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ba4dd7ef6674719ab49eaf7e9936230" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 } health_report { overall_health: UNKNOWN } } }
I20260807 02:31:05.768611 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.764979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.17.53.2:36603)
0807 02:31:05.765114 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:05.765117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:05.765131 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:31:05.765213 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:05.765215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:05.765273 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:05.765366 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:05.765638 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.766193 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.767185 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:05.767423 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:05.767511 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:05.767802 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:05.768214 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:05.768330 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:05.768443 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 02:31:05.768479 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:31:05.774991 13931 raft_consensus.cc:1261] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 1 FOLLOWER]: Refusing update from remote peer 8ba4dd7ef6674719ab49eaf7e9936230: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:05.775027 13526 raft_consensus.cc:1261] T 1dfd98d91597433bbfd625ddce4ca6f8 P ccf2556364584a80a09397879eda2174 [term 1 FOLLOWER]: Refusing update from remote peer 8ba4dd7ef6674719ab49eaf7e9936230: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:05.775722 14029 consensus_queue.cc:1035] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccf2556364584a80a09397879eda2174" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:05.775906 14034 consensus_queue.cc:1035] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb1d13bf4cf46d58c25a3ef1787acf3" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:05.784116 14043 mvcc.cc:205] Tried to move back new op lower bound from 7315742170206531584 to 7315742170075987968. Current Snapshot: MvccSnapshot[applied={T|T < 7315742170206531584}]
I20260807 02:31:05.793133 13311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:05.793316 13311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:05.795868 13311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16806a811669448dad6382a6537bc319: Sending DeleteTablet for 3 replicas of tablet 1dfd98d91597433bbfd625ddce4ca6f8
I20260807 02:31:05.796720 13498 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:05 UTC) from {username='slave'} at 127.0.0.1:37478
I20260807 02:31:05.796723 13895 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:05 UTC) from {username='slave'} at 127.0.0.1:35180
I20260807 02:31:05.796818 13694 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfd98d91597433bbfd625ddce4ca6f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:05 UTC) from {username='slave'} at 127.0.0.1:48412
I20260807 02:31:05.797344 14054 tablet_replica.cc:330] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: stopping tablet replica
I20260807 02:31:05.797392 14057 tablet_replica.cc:330] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: stopping tablet replica
I20260807 02:31:05.797734 14057 raft_consensus.cc:2229] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:31:05.798013 14054 raft_consensus.cc:2229] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:05.798214 14057 raft_consensus.cc:2258] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:05.798318 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 13401
I20260807 02:31:05.804162 14054 raft_consensus.cc:2258] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:05.808396 14057 ts_tablet_manager.cc:1905] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:05.812374 14057 ts_tablet_manager.cc:1918] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:05.812498 14057 log.cc:1196] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/1dfd98d91597433bbfd625ddce4ca6f8
I20260807 02:31:05.812896 14057 ts_tablet_manager.cc:1939] T 1dfd98d91597433bbfd625ddce4ca6f8 P 8ba4dd7ef6674719ab49eaf7e9936230: Deleting consensus metadata
I20260807 02:31:05.813874 14054 ts_tablet_manager.cc:1905] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:05.814358 13294 catalog_manager.cc:4841] TS 8ba4dd7ef6674719ab49eaf7e9936230 (127.17.53.2:36603): tablet 1dfd98d91597433bbfd625ddce4ca6f8 (table test-table [id=21fb7eabd8d044c8a222aa4d9790c5e4]) successfully deleted
I20260807 02:31:05.816694 14054 ts_tablet_manager.cc:1918] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:05.816905 14054 log.cc:1196] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/1dfd98d91597433bbfd625ddce4ca6f8
W20260807 02:31:05.817044 13295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:33579 (error 108)
I20260807 02:31:05.817301 14054 ts_tablet_manager.cc:1939] T 1dfd98d91597433bbfd625ddce4ca6f8 P 9bb1d13bf4cf46d58c25a3ef1787acf3: Deleting consensus metadata
I20260807 02:31:05.818034 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 13600
I20260807 02:31:05.818856 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:05.817930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33579 (local address 127.0.0.1:37490)
0807 02:31:05.818669 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:05.818753 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624}
I20260807 02:31:05.838757 13297 catalog_manager.cc:4841] TS 9bb1d13bf4cf46d58c25a3ef1787acf3 (127.17.53.3:44985): tablet 1dfd98d91597433bbfd625ddce4ca6f8 (table test-table [id=21fb7eabd8d044c8a222aa4d9790c5e4]) successfully deleted
W20260807 02:31:05.840931 13295 catalog_manager.cc:4584] TS ccf2556364584a80a09397879eda2174 (127.17.53.1:33579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dfd98d91597433bbfd625ddce4ca6f8: Network error: Client connection negotiation failed: client connection to 127.17.53.1:33579: connect: Connection refused (error 111)
I20260807 02:31:05.847566 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 13802
I20260807 02:31:05.860283 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 13271
2026-08-07T02:31:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1198 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:05 dist-test-slave-g170 krb5kdc[14062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:05 dist-test-slave-g170 krb5kdc[14062](info): set up 2 sockets
Aug 07 02:31:05 dist-test-slave-g170 krb5kdc[14062](info): commencing operation
krb5kdc: starting...
W20260807 02:31:07.122484 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:07.261895 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38577
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:38577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:07.378196 14083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:07.378515 14083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:07.378598 14083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:07.382587 14083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:07.382666 14083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:07.382704 14083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:07.382761 14083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:07.382800 14083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:07.387979 14083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:38577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:38577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:07.389075 14083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:07.390091 14083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:07.398713 14094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:07.398768 14093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:07.398730 14096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:07.398939 14083 server_base.cc:1034] running on GCE node
I20260807 02:31:07.399667 14083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:07.400709 14083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:07.401926 14083 hybrid_clock.cc:648] HybridClock initialized: now 1786069867401899 us; error 41 us; skew 500 ppm
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:07.404750 14083 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:07.405624 14083 webserver.cc:460] Webserver started at http://127.17.53.62:44951/ using document root <none> and password file <none>
I20260807 02:31:07.406150 14083 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:07.406208 14083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:07.406369 14083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:07.408314 14083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "72c0bf9751d549519b98fe5bba62ae3c"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "e003b475b3cf72dd10c9f52c5c9a8fe7"
server_key_iv: "49947613e9e8e0c90254e4dd2de03cc9"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.408895 14083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "72c0bf9751d549519b98fe5bba62ae3c"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "e003b475b3cf72dd10c9f52c5c9a8fe7"
server_key_iv: "49947613e9e8e0c90254e4dd2de03cc9"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.412478 14083 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 02:31:07.414983 14107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.416368 14083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 02:31:07.416548 14083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "72c0bf9751d549519b98fe5bba62ae3c"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "e003b475b3cf72dd10c9f52c5c9a8fe7"
server_key_iv: "49947613e9e8e0c90254e4dd2de03cc9"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.416672 14083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:07.445256 14083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:07.449131 14083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:07.449373 14083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:07.457722 14083 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:38577
I20260807 02:31:07.457763 14183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:38577 every 8 connection(s)
I20260807 02:31:07.458801 14083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:07.459241 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 14083
I20260807 02:31:07.459407 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:07.459719 17620 external_mini_cluster.cc:1428] Setting key ca299e5f99e558f73ae3df0676b0a5cd
I20260807 02:31:07.462136 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:07.469820 14184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: Bootstrap starting.
I20260807 02:31:07.472959 14184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:07.473774 14184 log.cc:824] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:07.475557 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:07.461276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.17.53.62:38577)
0807 02:31:07.461716 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:07.461738 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:07.461775 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:31:07.462500 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:07.462519 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:07.462854 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:07.463132 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:07.464171 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.465270 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.466570 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.467035 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.467154 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:07.470377 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:07.470384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:07.470400 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:07.470434 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:07.472546 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.473116 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.473129 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.473135 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.473265 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.473581 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.473584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.473585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.474027 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:07.474255 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:07.474544 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 02:31:07.474838 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":149,"threads_started":1}
I20260807 02:31:07.476014 14184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: No bootstrap required, opened a new log
I20260807 02:31:07.480674 14184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } }
I20260807 02:31:07.481002 14184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:07.481094 14184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c0bf9751d549519b98fe5bba62ae3c, State: Initialized, Role: FOLLOWER
I20260807 02:31:07.481540 14184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } }
I20260807 02:31:07.481703 14184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:07.481801 14184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:07.481918 14184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:07.482887 14184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } }
I20260807 02:31:07.483266 14184 leader_election.cc:302] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c0bf9751d549519b98fe5bba62ae3c; no voters:
I20260807 02:31:07.483606 14184 leader_election.cc:288] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:07.483796 14194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:07.484053 14194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [term 1 LEADER]: Becoming Leader. State: Replica: 72c0bf9751d549519b98fe5bba62ae3c, State: Running, Role: LEADER
I20260807 02:31:07.484375 14194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } }
I20260807 02:31:07.485060 14184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:07.486100 14195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c0bf9751d549519b98fe5bba62ae3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } } }
I20260807 02:31:07.486246 14195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:07.486677 14205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:07.487540 14196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c0bf9751d549519b98fe5bba62ae3c. Latest consensus state: current_term: 1 leader_uuid: "72c0bf9751d549519b98fe5bba62ae3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c0bf9751d549519b98fe5bba62ae3c" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 38577 } } }
I20260807 02:31:07.487650 14196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:07.490080 14205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:07.496065 14205 catalog_manager.cc:1347] Generated new cluster ID: 8d6f7fa0951a47e8a9d19d812d26eb32
I20260807 02:31:07.496138 14205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:07.504566 14205 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:07.505771 14205 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:07.514976 14205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: Generated new TSK 0
I20260807 02:31:07.515875 14205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.53.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
Entry for principal kudu/127.17.53.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab.
I20260807 02:31:07.550228 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:07.676391 14225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:07.676702 14225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:07.676820 14225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:07.680976 14225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:07.681067 14225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:07.681162 14225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:31:07.686586 14225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.1.keytab
--principal=kudu/127.17.53.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:07.687793 14225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:07.688907 14225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:07.698894 14231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:07.698894 14233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:07.698925 14230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:07.699358 14225 server_base.cc:1034] running on GCE node
I20260807 02:31:07.699769 14225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:07.700398 14225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:07.701586 14225 hybrid_clock.cc:648] HybridClock initialized: now 1786069867701554 us; error 40 us; skew 500 ppm
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:07.704741 14225 init.cc:377] Logged in from keytab as kudu/127.17.53.1@KRBTEST.COM (short username kudu)
I20260807 02:31:07.705678 14225 webserver.cc:460] Webserver started at http://127.17.53.1:40237/ using document root <none> and password file <none>
I20260807 02:31:07.706254 14225 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:07.706319 14225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:07.706493 14225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:07.708652 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "617708ff4e0d4edba29540001efe70aa"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "bede10f2fdbec0b5e8fdcf4d183be5cc"
server_key_iv: "c3e13c6b88439fdbe2fc4041cbd14e24"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.709309 14225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "617708ff4e0d4edba29540001efe70aa"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "bede10f2fdbec0b5e8fdcf4d183be5cc"
server_key_iv: "c3e13c6b88439fdbe2fc4041cbd14e24"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.713311 14225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:31:07.715852 14241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.717222 14225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:07.717372 14225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "617708ff4e0d4edba29540001efe70aa"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "bede10f2fdbec0b5e8fdcf4d183be5cc"
server_key_iv: "c3e13c6b88439fdbe2fc4041cbd14e24"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.717489 14225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:07.731542 14225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:07.735021 14225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:07.735256 14225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:07.735919 14225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:07.736891 14225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:07.736974 14225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.737049 14225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:07.737085 14225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.747076 14225 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:38351
I20260807 02:31:07.747116 14442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:38351 every 8 connection(s)
I20260807 02:31:07.748067 14225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:31:07.748579 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 14225
I20260807 02:31:07.748718 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:31:07.749101 17620 external_mini_cluster.cc:1428] Setting key 94f43ad8d794ea9fc2d7e5673211cfe6
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.1@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.2@KRBTEST.COM" created.
I20260807 02:31:07.766316 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:07.750181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:59501 (local address 127.17.53.62:38577)
0807 02:31:07.750339 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:07.750344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:07.751528 (+ 1184us) server_negotiation.cc:407] Connection header received
0807 02:31:07.753604 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:07.753609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:07.753680 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:07.753790 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:07.755191 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.755758 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.757768 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.758233 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.758413 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:07.761489 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:07.761496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:07.761502 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:07.761540 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:07.763205 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.763912 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.763916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.763917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.764010 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.764620 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.764625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.764627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.764958 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:07.765067 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:07.766025 (+ 958us) server_negotiation.cc:299] Negotiation successful
0807 02:31:07.766166 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:31:07.767643 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:07.750724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38577 (local address 127.17.53.1:59501)
0807 02:31:07.751331 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:07.751401 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:07.753285 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:07.753965 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:07.753972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:07.754408 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:07.755005 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:07.755032 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.755916 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.755923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:07.757571 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:07.757585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.758435 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.758439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:07.758613 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:07.759427 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:07.759445 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:07.761272 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:07.763339 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:07.763345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:07.763361 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:07.763662 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:07.764106 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:07.764108 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:07.764109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:07.764473 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:07.765106 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:07.765116 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:07.765591 (+ 475us) client_negotiation.cc:769] Sending connection context
0807 02:31:07.765931 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 02:31:07.766439 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":137,"threads_started":1}
I20260807 02:31:07.769215 14444 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38577
I20260807 02:31:07.769516 14444 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:07.770211 14444 heartbeater.cc:511] Master 127.17.53.62:38577 requested a full tablet report, sending...
I20260807 02:31:07.772148 14130 ts_manager.cc:194] Registered new tserver with Master: 617708ff4e0d4edba29540001efe70aa (127.17.53.1:38351)
I20260807 02:31:07.773522 14130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.1@KRBTEST.COM'} at 127.17.53.1:59501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
Entry for principal kudu/127.17.53.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab.
I20260807 02:31:07.784154 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:07.900475 14451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:07.900748 14451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:07.900838 14451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:07.904904 14451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:07.904963 14451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:07.905050 14451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:31:07.910379 14451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.2.keytab
--principal=kudu/127.17.53.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:07.911494 14451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:07.912549 14451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:07.922605 14461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:07.922684 14451 server_base.cc:1034] running on GCE node
W20260807 02:31:07.922626 14463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:07.922591 14459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:07.923377 14451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:07.924022 14451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:07.925231 14451 hybrid_clock.cc:648] HybridClock initialized: now 1786069867925215 us; error 30 us; skew 500 ppm
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:07.928403 14451 init.cc:377] Logged in from keytab as kudu/127.17.53.2@KRBTEST.COM (short username kudu)
I20260807 02:31:07.929425 14451 webserver.cc:460] Webserver started at http://127.17.53.2:44455/ using document root <none> and password file <none>
I20260807 02:31:07.930048 14451 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:07.930114 14451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:07.930361 14451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:07.932725 14451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "ad84e1d2a13d4408bb87878a76459588"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "3ee43e41fcea45c9815532cb34bae561"
server_key_iv: "a390461468d61a15d48602790592f2ba"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.933425 14451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "ad84e1d2a13d4408bb87878a76459588"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "3ee43e41fcea45c9815532cb34bae561"
server_key_iv: "a390461468d61a15d48602790592f2ba"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.937438 14451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:31:07.940044 14472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.941459 14451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:07.941610 14451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "ad84e1d2a13d4408bb87878a76459588"
format_stamp: "Formatted at 2026-08-07 02:31:07 on dist-test-slave-g170"
server_key: "3ee43e41fcea45c9815532cb34bae561"
server_key_iv: "a390461468d61a15d48602790592f2ba"
server_key_version: "encryptionkey@0"
I20260807 02:31:07.941735 14451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:07.961819 14451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:07.965720 14451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:07.965962 14451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:07.966660 14451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:07.967650 14451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:07.967799 14451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.967888 14451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:07.967948 14451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:07.979079 14451 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:36335
I20260807 02:31:07.979107 14644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:36335 every 8 connection(s)
I20260807 02:31:07.980114 14451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:31:07.982679 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 14451
I20260807 02:31:07.982822 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:31:07.983186 17620 external_mini_cluster.cc:1428] Setting key 14ce146bd6c06fe3ab7f18e11e90cf4b
Aug 07 02:31:07 dist-test-slave-g170 krb5kdc[14062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.2@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.3@KRBTEST.COM; defaulting to no policy
I20260807 02:31:07.996929 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:07.982307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:60507 (local address 127.17.53.62:38577)
0807 02:31:07.982484 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:07.982495 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:07.983524 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 02:31:07.985296 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:07.985299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:07.985346 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:07.985416 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:07.987065 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.988015 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.989333 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.989737 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.989874 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:07.992846 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:07.992852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:07.992854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:07.992884 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:07.994633 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.995215 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.995224 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.995225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.995306 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:07.995675 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:07.995678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:07.995680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:07.995873 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:07.995993 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:07.996604 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 02:31:07.996734 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.3@KRBTEST.COM" created.
I20260807 02:31:07.997985 14650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:07.982753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38577 (local address 127.17.53.2:60507)
0807 02:31:07.983350 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:07.983415 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:07.984995 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:07.985613 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:07.985626 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:07.986162 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:07.986884 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:07.986907 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.988148 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.988151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:07.989203 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:07.989214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:07.989909 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:07.989912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:07.990053 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:07.990859 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:07.990882 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:07.992636 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:07.994767 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:07.994772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:07.994784 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:07.995035 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:07.995410 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:07.995414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:07.995416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:07.995568 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:07.996031 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:07.996039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:07.996330 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 02:31:07.996556 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 02:31:07.996941 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":151,"threads_started":1}
I20260807 02:31:07.999495 14646 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38577
I20260807 02:31:07.999804 14646 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:08.000361 14646 heartbeater.cc:511] Master 127.17.53.62:38577 requested a full tablet report, sending...
I20260807 02:31:08.001524 14130 ts_manager.cc:194] Registered new tserver with Master: ad84e1d2a13d4408bb87878a76459588 (127.17.53.2:36335)
I20260807 02:31:08.002115 14130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.2@KRBTEST.COM'} at 127.17.53.2:60507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
Entry for principal kudu/127.17.53.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab.
I20260807 02:31:08.016485 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:08.134814 14654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:08.135161 14654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:08.135238 14654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:08.139393 14654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:08.139490 14654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:08.139593 14654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:08.145234 14654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.3.keytab
--principal=kudu/127.17.53.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:08.146420 14654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:08.147477 14654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:08.157727 14670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.157727 14667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.157755 14664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:08.158005 14654 server_base.cc:1034] running on GCE node
I20260807 02:31:08.158520 14654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:08.159323 14654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:08.160547 14654 hybrid_clock.cc:648] HybridClock initialized: now 1786069868160530 us; error 27 us; skew 500 ppm
Aug 07 02:31:08 dist-test-slave-g170 krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069868, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:08.163843 14654 init.cc:377] Logged in from keytab as kudu/127.17.53.3@KRBTEST.COM (short username kudu)
I20260807 02:31:08.164940 14654 webserver.cc:460] Webserver started at http://127.17.53.3:36343/ using document root <none> and password file <none>
I20260807 02:31:08.165531 14654 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:08.165594 14654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:08.165784 14654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:08.167950 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "3cf0b9b0ae3f46c5b23047688977ea21"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "d4fb3374fa03191f160a38a18b78acf3"
server_key_iv: "b396e8205a3eb142a6a3a38806b408b8"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.168500 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "3cf0b9b0ae3f46c5b23047688977ea21"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "d4fb3374fa03191f160a38a18b78acf3"
server_key_iv: "b396e8205a3eb142a6a3a38806b408b8"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.172631 14654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 02:31:08.175465 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.176860 14654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:08.176993 14654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "3cf0b9b0ae3f46c5b23047688977ea21"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "d4fb3374fa03191f160a38a18b78acf3"
server_key_iv: "b396e8205a3eb142a6a3a38806b408b8"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.177105 14654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:08.203419 14654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:08.206881 14654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:08.207105 14654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:08.207772 14654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:08.208660 14654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:08.208710 14654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.208799 14654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:08.208837 14654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.218858 14654 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:32951
I20260807 02:31:08.218874 14868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:32951 every 8 connection(s)
I20260807 02:31:08.219897 14654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:08.224920 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 14654
I20260807 02:31:08.225091 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:08.225381 17620 external_mini_cluster.cc:1428] Setting key fed1195ed02933353c20128ba15286d9
Aug 07 02:31:08 dist-test-slave-g170 krb5kdc[14062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069868, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.3@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:08.235447 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.222217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:46757 (local address 127.17.53.62:38577)
0807 02:31:08.222385 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.222389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.223346 (+ 957us) server_negotiation.cc:407] Connection header received
0807 02:31:08.224787 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.224792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.224860 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.224957 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:08.226624 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.227351 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.228740 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.229032 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.229143 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.231807 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:08.231811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:08.231813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:08.231838 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:08.233361 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:08.233880 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:08.233884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:08.233885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:08.233964 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:08.234366 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:08.234368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:08.234370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:08.234557 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:08.234635 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.235207 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.235322 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:31:08.236328 14873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.222657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:38577 (local address 127.17.53.3:46757)
0807 02:31:08.223166 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.223213 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.224502 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.225193 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.225204 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.225658 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:08.226442 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.226466 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.227541 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.227544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.228598 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.228609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.229206 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.229210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.229400 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.230005 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:08.230023 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:08.231574 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:08.233472 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:08.233478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:08.233491 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:08.233772 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:08.234092 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:08.234094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:08.234096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:08.234244 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:08.234718 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:08.234724 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:08.234966 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.235135 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.235407 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
I20260807 02:31:08.237599 14869 heartbeater.cc:347] Connected to a master server at 127.17.53.62:38577
I20260807 02:31:08.237883 14869 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:08.238498 14869 heartbeater.cc:511] Master 127.17.53.62:38577 requested a full tablet report, sending...
I20260807 02:31:08.239766 14132 ts_manager.cc:194] Registered new tserver with Master: 3cf0b9b0ae3f46c5b23047688977ea21 (127.17.53.3:32951)
I20260807 02:31:08.240417 14132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.53.3@KRBTEST.COM'} at 127.17.53.3:46757
I20260807 02:31:08.251880 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:08.262421 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.254195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.17.53.62:38577)
0807 02:31:08.254373 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.254378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.254525 (+ 147us) server_negotiation.cc:407] Connection header received
0807 02:31:08.254635 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.254638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.254684 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.254758 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:08.255564 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.256156 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.257142 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.257377 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.257465 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.259025 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:08.259028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:08.259031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:08.259056 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:08.260453 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:08.260968 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:08.260971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:08.260973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:08.261053 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:08.261392 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:08.261395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:08.261396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:08.261618 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:08.261766 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.262056 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.262211 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 02:31:08.265707 14132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:08.269332 14132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:08.286383 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.280745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.17.53.2:36335)
0807 02:31:08.281146 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.281158 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.281184 (+ 26us) server_negotiation.cc:407] Connection header received
0807 02:31:08.281296 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.281302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.281539 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.281715 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:08.282010 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283177 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.284891 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.285668 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.285795 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.286049 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.286103 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.286209 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":136,"threads_started":1}
I20260807 02:31:08.286381 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.280411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:36335 (local address 127.0.0.1:45774)
0807 02:31:08.280873 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.280890 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.280985 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.281723 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.281729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.281754 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:08.281887 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.281893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283287 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.283290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.284718 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.284729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.285795 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.285798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.285910 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.285949 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.286055 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.286215 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":94,"thread_start_us":108,"threads_started":1}
I20260807 02:31:08.286406 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.279836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38351 (local address 127.0.0.1:58222)
0807 02:31:08.280209 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.280265 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.280401 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.282230 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.282233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.282349 (+ 116us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:08.282434 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.282441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283812 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.283818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.284603 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.284612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.285888 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.285892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.285980 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.286008 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.286080 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.286230 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260807 02:31:08.287259 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.279940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.17.53.1:38351)
0807 02:31:08.281734 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.281741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.281763 (+ 22us) server_negotiation.cc:407] Connection header received
0807 02:31:08.281843 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.281848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.282065 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.282592 (+ 527us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:08.282732 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283714 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.284732 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.285642 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.286450 (+ 808us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.286629 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.286690 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.286777 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1680,"thread_start_us":98,"threads_started":1}
I20260807 02:31:08.287878 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.280593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:32951 (local address 127.0.0.1:41772)
0807 02:31:08.281333 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.281346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.281430 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.282057 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.282060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.282077 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:08.282172 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.282179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283232 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.283238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.285256 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.285266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.286840 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.286844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.286947 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.286980 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.287675 (+ 695us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.287757 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":67,"thread_start_us":104,"threads_started":1}
I20260807 02:31:08.287943 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.280922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.17.53.3:32951)
0807 02:31:08.281619 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.281629 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.281676 (+ 47us) server_negotiation.cc:407] Connection header received
0807 02:31:08.281749 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.281754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.281940 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.282086 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:08.282339 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.283097 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.285396 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.286377 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.287266 (+ 889us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.287556 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.287709 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.287801 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":107,"threads_started":1}
I20260807 02:31:08.289753 14545 tablet_service.cc:1467] Processing CreateTablet for tablet 19ace16982a24b3c914cd93a2d64b96d (DEFAULT_TABLE table=test-table [id=933bf6b79704433eb9f196f97aebc2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:08.290465 14763 tablet_service.cc:1467] Processing CreateTablet for tablet 19ace16982a24b3c914cd93a2d64b96d (DEFAULT_TABLE table=test-table [id=933bf6b79704433eb9f196f97aebc2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:08.290969 14545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ace16982a24b3c914cd93a2d64b96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:08.290812 14326 tablet_service.cc:1467] Processing CreateTablet for tablet 19ace16982a24b3c914cd93a2d64b96d (DEFAULT_TABLE table=test-table [id=933bf6b79704433eb9f196f97aebc2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:08.291487 14763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ace16982a24b3c914cd93a2d64b96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:08.291822 14326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19ace16982a24b3c914cd93a2d64b96d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:08.298214 14896 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Bootstrap starting.
I20260807 02:31:08.299225 14898 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: Bootstrap starting.
I20260807 02:31:08.300506 14896 tablet_bootstrap.cc:654] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:08.301095 14898 tablet_bootstrap.cc:654] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:08.301136 14900 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Bootstrap starting.
I20260807 02:31:08.301709 14896 log.cc:824] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:08.301934 14898 log.cc:824] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:08.303611 14900 tablet_bootstrap.cc:654] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:08.304107 14896 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: No bootstrap required, opened a new log
I20260807 02:31:08.304208 14898 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: No bootstrap required, opened a new log
I20260807 02:31:08.304316 14896 ts_tablet_manager.cc:1397] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:31:08.304427 14898 ts_tablet_manager.cc:1397] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 02:31:08.304425 14900 log.cc:824] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:08.306427 14900 tablet_bootstrap.cc:492] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: No bootstrap required, opened a new log
I20260807 02:31:08.306599 14900 ts_tablet_manager.cc:1397] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 02:31:08.308037 14898 raft_consensus.cc:348] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.308276 14898 raft_consensus.cc:374] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:08.308332 14898 raft_consensus.cc:729] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617708ff4e0d4edba29540001efe70aa, State: Initialized, Role: FOLLOWER
I20260807 02:31:08.308205 14896 raft_consensus.cc:348] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.308473 14896 raft_consensus.cc:374] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:08.308542 14896 raft_consensus.cc:729] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf0b9b0ae3f46c5b23047688977ea21, State: Initialized, Role: FOLLOWER
I20260807 02:31:08.308892 14898 consensus_queue.cc:260] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.308981 14896 consensus_queue.cc:260] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.309547 14444 heartbeater.cc:503] Master 127.17.53.62:38577 was elected leader, sending a full tablet report...
I20260807 02:31:08.309857 14898 ts_tablet_manager.cc:1428] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 02:31:08.309890 14896 ts_tablet_manager.cc:1428] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 02:31:08.310225 14869 heartbeater.cc:503] Master 127.17.53.62:38577 was elected leader, sending a full tablet report...
I20260807 02:31:08.311453 14900 raft_consensus.cc:348] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.311690 14900 raft_consensus.cc:374] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:08.311753 14900 raft_consensus.cc:729] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad84e1d2a13d4408bb87878a76459588, State: Initialized, Role: FOLLOWER
I20260807 02:31:08.312170 14900 consensus_queue.cc:260] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.312855 14646 heartbeater.cc:503] Master 127.17.53.62:38577 was elected leader, sending a full tablet report...
I20260807 02:31:08.313141 14900 ts_tablet_manager.cc:1428] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 02:31:08.342314 14907 raft_consensus.cc:482] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:08.342530 14907 raft_consensus.cc:504] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.343766 14907 leader_election.cc:288] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cf0b9b0ae3f46c5b23047688977ea21 (127.17.53.3:32951), ad84e1d2a13d4408bb87878a76459588 (127.17.53.2:36335)
I20260807 02:31:08.348433 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.344071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:42843 (local address 127.17.53.3:32951)
0807 02:31:08.344225 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.344232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.344484 (+ 252us) server_negotiation.cc:407] Connection header received
0807 02:31:08.344656 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.344662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.344732 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.344886 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:08.345494 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.346028 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.347270 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.347741 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.347858 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.347931 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.348200 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.348308 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:31:08.348434 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.343952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:32951 (local address 127.17.53.1:42843)
0807 02:31:08.344357 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.344377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.344516 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.344925 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.344932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.344953 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:08.345252 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.345287 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.346215 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.346224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.347119 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.347129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.347891 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.347899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.348014 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.348051 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.348158 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.348236 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":20,"spinlock_wait_cycles":7936,"thread_start_us":153,"threads_started":1}
I20260807 02:31:08.348541 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.344336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:59597 (local address 127.17.53.2:36335)
0807 02:31:08.344514 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.344519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.344570 (+ 51us) server_negotiation.cc:407] Connection header received
0807 02:31:08.344743 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.344746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.344860 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.344945 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:08.345460 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.346028 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.347324 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.347791 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.347931 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.348001 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.348322 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.348414 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 02:31:08.349045 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.344069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:36335 (local address 127.17.53.1:59597)
0807 02:31:08.344477 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.344492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.344625 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.344987 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.344993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.345031 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:08.345282 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.345289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.346150 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.346156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.347176 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.347187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.348027 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.348031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.348163 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.348198 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.348822 (+ 624us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.348895 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":111,"thread_start_us":120,"threads_started":1}
I20260807 02:31:08.349318 14794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ace16982a24b3c914cd93a2d64b96d" candidate_uuid: "617708ff4e0d4edba29540001efe70aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" is_pre_election: true
I20260807 02:31:08.349644 14794 raft_consensus.cc:2454] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 617708ff4e0d4edba29540001efe70aa in term 0.
I20260807 02:31:08.349685 14574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ace16982a24b3c914cd93a2d64b96d" candidate_uuid: "617708ff4e0d4edba29540001efe70aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad84e1d2a13d4408bb87878a76459588" is_pre_election: true
I20260807 02:31:08.349982 14574 raft_consensus.cc:2454] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 617708ff4e0d4edba29540001efe70aa in term 0.
I20260807 02:31:08.350230 14247 leader_election.cc:302] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cf0b9b0ae3f46c5b23047688977ea21, 617708ff4e0d4edba29540001efe70aa; no voters:
I20260807 02:31:08.350541 14907 raft_consensus.cc:2781] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:08.350651 14907 raft_consensus.cc:482] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:08.350721 14907 raft_consensus.cc:3037] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:08.351878 14907 raft_consensus.cc:504] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.352245 14907 leader_election.cc:288] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [CANDIDATE]: Term 1 election: Requested vote from peers 3cf0b9b0ae3f46c5b23047688977ea21 (127.17.53.3:32951), ad84e1d2a13d4408bb87878a76459588 (127.17.53.2:36335)
I20260807 02:31:08.352658 14794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ace16982a24b3c914cd93a2d64b96d" candidate_uuid: "617708ff4e0d4edba29540001efe70aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf0b9b0ae3f46c5b23047688977ea21"
I20260807 02:31:08.352681 14574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19ace16982a24b3c914cd93a2d64b96d" candidate_uuid: "617708ff4e0d4edba29540001efe70aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad84e1d2a13d4408bb87878a76459588"
I20260807 02:31:08.352835 14794 raft_consensus.cc:3037] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:08.352835 14574 raft_consensus.cc:3037] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:08.354274 14794 raft_consensus.cc:2454] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617708ff4e0d4edba29540001efe70aa in term 1.
I20260807 02:31:08.354282 14574 raft_consensus.cc:2454] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617708ff4e0d4edba29540001efe70aa in term 1.
I20260807 02:31:08.354650 14242 leader_election.cc:302] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617708ff4e0d4edba29540001efe70aa, ad84e1d2a13d4408bb87878a76459588; no voters:
I20260807 02:31:08.354923 14907 raft_consensus.cc:2781] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:08.355191 14907 raft_consensus.cc:686] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [term 1 LEADER]: Becoming Leader. State: Replica: 617708ff4e0d4edba29540001efe70aa, State: Running, Role: LEADER
I20260807 02:31:08.355543 14907 consensus_queue.cc:237] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } }
I20260807 02:31:08.359109 14132 catalog_manager.cc:5495] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 617708ff4e0d4edba29540001efe70aa (127.17.53.1). New cstate: current_term: 1 leader_uuid: "617708ff4e0d4edba29540001efe70aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617708ff4e0d4edba29540001efe70aa" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 38351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 } health_report { overall_health: UNKNOWN } } }
I20260807 02:31:08.372637 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.369113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.17.53.1:38351)
0807 02:31:08.369251 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.369255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.369268 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:31:08.369390 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.369393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.369440 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.369570 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:08.369880 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.370371 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.371200 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.371417 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.371495 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.371816 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:08.372200 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:08.372309 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.372430 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.372478 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:31:08.378407 14574 raft_consensus.cc:1261] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 1 FOLLOWER]: Refusing update from remote peer 617708ff4e0d4edba29540001efe70aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:08.378407 14794 raft_consensus.cc:1261] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 1 FOLLOWER]: Refusing update from remote peer 617708ff4e0d4edba29540001efe70aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:08.379140 14907 consensus_queue.cc:1035] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad84e1d2a13d4408bb87878a76459588" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 36335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:08.379357 14914 consensus_queue.cc:1035] T 19ace16982a24b3c914cd93a2d64b96d P 617708ff4e0d4edba29540001efe70aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf0b9b0ae3f46c5b23047688977ea21" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:08.387665 14922 mvcc.cc:205] Tried to move back new op lower bound from 7315742180870397952 to 7315742180787449856. Current Snapshot: MvccSnapshot[applied={T|T < 7315742180870397952}]
I20260807 02:31:08.395880 14131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:08.396060 14131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:08.398524 14131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72c0bf9751d549519b98fe5bba62ae3c: Sending DeleteTablet for 3 replicas of tablet 19ace16982a24b3c914cd93a2d64b96d
I20260807 02:31:08.399320 14545 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ace16982a24b3c914cd93a2d64b96d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:08 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:45774
I20260807 02:31:08.399410 14326 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ace16982a24b3c914cd93a2d64b96d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:08 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:58222
I20260807 02:31:08.399816 14763 tablet_service.cc:1514] Processing DeleteTablet for tablet 19ace16982a24b3c914cd93a2d64b96d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:08 UTC) from {username='kudu', principal='kudu/127.17.53.62@KRBTEST.COM'} at 127.0.0.1:41772
I20260807 02:31:08.399895 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 14225
I20260807 02:31:08.400552 14936 tablet_replica.cc:330] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: stopping tablet replica
I20260807 02:31:08.400969 14936 raft_consensus.cc:2229] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:08.401341 14936 raft_consensus.cc:2258] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:08.402737 14931 tablet_replica.cc:330] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: stopping tablet replica
I20260807 02:31:08.403071 14931 raft_consensus.cc:2229] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:08.403427 14931 raft_consensus.cc:2258] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:08.408484 14936 ts_tablet_manager.cc:1905] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:08.410789 14936 ts_tablet_manager.cc:1918] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:08.410874 14936 log.cc:1196] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/19ace16982a24b3c914cd93a2d64b96d
I20260807 02:31:08.411230 14936 ts_tablet_manager.cc:1939] T 19ace16982a24b3c914cd93a2d64b96d P 3cf0b9b0ae3f46c5b23047688977ea21: Deleting consensus metadata
I20260807 02:31:08.412099 14931 ts_tablet_manager.cc:1905] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:08.412400 14115 catalog_manager.cc:4841] TS 3cf0b9b0ae3f46c5b23047688977ea21 (127.17.53.3:32951): tablet 19ace16982a24b3c914cd93a2d64b96d (table test-table [id=933bf6b79704433eb9f196f97aebc2df]) successfully deleted
I20260807 02:31:08.415933 14931 ts_tablet_manager.cc:1918] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:08.416071 14931 log.cc:1196] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/19ace16982a24b3c914cd93a2d64b96d
I20260807 02:31:08.416509 14931 ts_tablet_manager.cc:1939] T 19ace16982a24b3c914cd93a2d64b96d P ad84e1d2a13d4408bb87878a76459588: Deleting consensus metadata
I20260807 02:31:08.417660 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 14451
W20260807 02:31:08.417661 14115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:38351 (error 108)
I20260807 02:31:08.421324 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.420940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:38351 (local address 127.0.0.1:58250)
0807 02:31:08.421112 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.421202 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 02:31:08.421540 14115 catalog_manager.cc:4584] TS 617708ff4e0d4edba29540001efe70aa (127.17.53.1:38351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19ace16982a24b3c914cd93a2d64b96d: Network error: Client connection negotiation failed: client connection to 127.17.53.1:38351: connect: Connection refused (error 111)
I20260807 02:31:08.426586 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 14654
I20260807 02:31:08.427057 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.426708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:36335 (local address 127.0.0.1:45790)
0807 02:31:08.426905 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.426963 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.2:36335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 02:31:08.440138 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 14083
2026-08-07T02:31:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2581 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T02:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:08Z Disabled control of system clock
I20260807 02:31:08.479959 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34087
--webserver_interface=127.17.53.62
--webserver_port=0
--builtin_ntp_servers=127.17.53.20:42461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:08.600093 14941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:08.600355 14941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:08.600389 14941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:08.604300 14941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:08.604351 14941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:08.604367 14941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:08.604383 14941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:08.604398 14941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:08.609181 14941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:34087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:08.610160 14941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:08.611483 14941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:08.620221 14954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.620222 14956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.620222 14953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:08.620735 14941 server_base.cc:1034] running on GCE node
I20260807 02:31:08.621331 14941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:08.622447 14941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:08.623651 14941 hybrid_clock.cc:648] HybridClock initialized: now 1786069868623631 us; error 32 us; skew 500 ppm
I20260807 02:31:08.625782 14941 webserver.cc:460] Webserver started at http://127.17.53.62:39535/ using document root <none> and password file <none>
I20260807 02:31:08.626446 14941 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:08.626545 14941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:08.626796 14941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:08.628957 14941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "1a8e8c5f57024cadb02d0a107db16142"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "b28122b679f71863e5bd68538fdbdeea"
server_key_iv: "b13147e2cb54c288410071b6ddb91c64"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.629539 14941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "1a8e8c5f57024cadb02d0a107db16142"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "b28122b679f71863e5bd68538fdbdeea"
server_key_iv: "b13147e2cb54c288410071b6ddb91c64"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.633329 14941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:31:08.635818 14965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.637140 14941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:08.637300 14941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "1a8e8c5f57024cadb02d0a107db16142"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "b28122b679f71863e5bd68538fdbdeea"
server_key_iv: "b13147e2cb54c288410071b6ddb91c64"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.637415 14941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:08.676033 14941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:08.676939 14941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:08.677155 14941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:08.687055 14941 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:34087
I20260807 02:31:08.687070 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:34087 every 8 connection(s)
I20260807 02:31:08.688047 14941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:08.691363 15039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:08.697072 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: Bootstrap starting.
I20260807 02:31:08.697294 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 14941
I20260807 02:31:08.697444 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:08.697726 17620 external_mini_cluster.cc:1428] Setting key 98ab089c53dd3249cf974279a5f1f4c0
I20260807 02:31:08.699888 15039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:08.700659 15039 log.cc:824] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:08.702574 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: No bootstrap required, opened a new log
I20260807 02:31:08.706166 15039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } }
I20260807 02:31:08.706492 15039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:08.706575 15039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8e8c5f57024cadb02d0a107db16142, State: Initialized, Role: FOLLOWER
I20260807 02:31:08.707006 15039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } }
I20260807 02:31:08.707126 15039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:08.707223 15039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:08.707346 15039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:08.707468 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.699115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.17.53.62:34087)
0807 02:31:08.699823 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.699838 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.699874 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:31:08.700574 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.700592 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.701071 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.701447 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:08.701881 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.702911 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.703820 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.704302 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.704480 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.705391 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:08.705396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:08.705411 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:08.705802 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:08.705856 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:08.705877 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:08.705989 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.706130 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.706433 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
I20260807 02:31:08.708268 15039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } }
I20260807 02:31:08.708643 15039 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8e8c5f57024cadb02d0a107db16142; no voters:
I20260807 02:31:08.708994 15039 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:08.709108 15049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:08.709369 15049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8e8c5f57024cadb02d0a107db16142, State: Running, Role: LEADER
I20260807 02:31:08.709722 15049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } }
I20260807 02:31:08.710263 15039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:08.710898 15051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8e8c5f57024cadb02d0a107db16142. Latest consensus state: current_term: 1 leader_uuid: "1a8e8c5f57024cadb02d0a107db16142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } } }
I20260807 02:31:08.711023 15051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:08.711346 15052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8e8c5f57024cadb02d0a107db16142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8e8c5f57024cadb02d0a107db16142" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 34087 } } }
I20260807 02:31:08.711457 15052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:08.711735 15058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:08.714607 15058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:08.720518 15058 catalog_manager.cc:1347] Generated new cluster ID: bfce69b59e4344a19f016ba9f23ddbd6
I20260807 02:31:08.720599 15058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:08.730580 15058 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:08.731539 15058 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:08.737336 15058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: Generated new TSK 0
I20260807 02:31:08.738162 15058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:08.742527 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.1:0
--local_ip_for_outbound_sockets=127.17.53.1
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--builtin_ntp_servers=127.17.53.20:42461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 02:31:08.865057 15078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:08.865331 15078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:08.865371 15078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:08.869468 15078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:08.869521 15078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:08.869580 15078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.1
I20260807 02:31:08.874519 15078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.53.1
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.1
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:08.875548 15078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:08.877359 15078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:08.887620 15092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.887588 15087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:08.887650 15089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:08.887804 15078 server_base.cc:1034] running on GCE node
I20260807 02:31:08.888334 15078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:08.889043 15078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:08.890231 15078 hybrid_clock.cc:648] HybridClock initialized: now 1786069868890215 us; error 29 us; skew 500 ppm
I20260807 02:31:08.892408 15078 webserver.cc:460] Webserver started at http://127.17.53.1:45155/ using document root <none> and password file <none>
I20260807 02:31:08.893178 15078 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:08.893257 15078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:08.893494 15078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:08.895768 15078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data/instance:
uuid: "f3a55f409cea42238e37dd9666214dd5"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "ba34cbe341fa4d01d061dd4d29cc9192"
server_key_iv: "1282ac5d9eba57cf8613e7ed83ebbe4c"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.896385 15078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance:
uuid: "f3a55f409cea42238e37dd9666214dd5"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "ba34cbe341fa4d01d061dd4d29cc9192"
server_key_iv: "1282ac5d9eba57cf8613e7ed83ebbe4c"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.900674 15078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 02:31:08.903570 15101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.905052 15078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:08.905205 15078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal
uuid: "f3a55f409cea42238e37dd9666214dd5"
format_stamp: "Formatted at 2026-08-07 02:31:08 on dist-test-slave-g170"
server_key: "ba34cbe341fa4d01d061dd4d29cc9192"
server_key_iv: "1282ac5d9eba57cf8613e7ed83ebbe4c"
server_key_version: "encryptionkey@0"
I20260807 02:31:08.905337 15078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:08.930110 15078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:08.930863 15078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:08.931063 15078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:08.931664 15078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:08.932631 15078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:08.932705 15078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.932796 15078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:08.932835 15078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:08.942847 15078 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.1:33461
I20260807 02:31:08.942880 15306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.1:33461 every 8 connection(s)
I20260807 02:31:08.943811 15078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/data/info.pb
I20260807 02:31:08.951378 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 15078
I20260807 02:31:08.951514 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-0/wal/instance
I20260807 02:31:08.951846 17620 external_mini_cluster.cc:1428] Setting key 901ee1c96bd0672bfa4bf76703e6bbb8
I20260807 02:31:08.954352 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.2:0
--local_ip_for_outbound_sockets=127.17.53.2
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--builtin_ntp_servers=127.17.53.20:42461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:08.954632 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.945801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.1:52591 (local address 127.17.53.62:34087)
0807 02:31:08.945976 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:08.945981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:08.947088 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 02:31:08.948681 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:08.948685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:08.948749 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:08.948879 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:08.949885 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.950495 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.952199 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.952436 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.952567 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.953963 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:08.953966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:08.953968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:08.953984 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:08.954004 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:08.954014 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:08.954099 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:08.954362 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 02:31:08.954467 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:31:08.955754 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:08.946191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34087 (local address 127.17.53.1:52591)
0807 02:31:08.946918 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:08.946964 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:08.948433 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:08.949085 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:08.949092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:08.949188 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:08.949707 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.949731 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.950642 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.950649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.952039 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:08.952052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:08.952611 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:08.952615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:08.952850 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:08.953567 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:08.953569 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:08.953621 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:08.953765 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:08.954137 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:08.954149 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:08.954158 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 02:31:08.954317 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 02:31:08.954690 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":110,"threads_started":1}
I20260807 02:31:08.957201 15307 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34087
I20260807 02:31:08.957460 15307 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:08.957986 15307 heartbeater.cc:511] Master 127.17.53.62:34087 requested a full tablet report, sending...
I20260807 02:31:08.959556 14989 ts_manager.cc:194] Registered new tserver with Master: f3a55f409cea42238e37dd9666214dd5 (127.17.53.1:33461)
I20260807 02:31:08.960812 14989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.1:52591
W20260807 02:31:09.075474 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:09.075793 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:09.075878 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:09.080029 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:09.080118 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:09.080219 15311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.2
I20260807 02:31:09.085157 15311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.53.2
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.2
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:09.086277 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:09.087700 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:09.097628 15319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:09.097678 15323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:09.097666 15321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:09.097985 15311 server_base.cc:1034] running on GCE node
I20260807 02:31:09.098421 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:09.099045 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:09.100242 15311 hybrid_clock.cc:648] HybridClock initialized: now 1786069869100221 us; error 34 us; skew 500 ppm
I20260807 02:31:09.102197 15311 webserver.cc:460] Webserver started at http://127.17.53.2:43321/ using document root <none> and password file <none>
I20260807 02:31:09.102830 15311 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:09.102928 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:09.103193 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:09.105415 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data/instance:
uuid: "b3bee311cf92428b864a9ec9ddbdc54f"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "a4a0f09dad6ba485811fd401d53794fd"
server_key_iv: "cdc1297d285233e10d71a9d6399dc011"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.105984 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance:
uuid: "b3bee311cf92428b864a9ec9ddbdc54f"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "a4a0f09dad6ba485811fd401d53794fd"
server_key_iv: "cdc1297d285233e10d71a9d6399dc011"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.109798 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:31:09.112799 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.114087 15311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:09.114256 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal
uuid: "b3bee311cf92428b864a9ec9ddbdc54f"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "a4a0f09dad6ba485811fd401d53794fd"
server_key_iv: "cdc1297d285233e10d71a9d6399dc011"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.114372 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:09.143221 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:09.143945 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:09.144153 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:09.144832 15311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:09.145799 15311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:09.145881 15311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.145973 15311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:09.146025 15311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.155542 15311 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.2:45099
I20260807 02:31:09.155575 15511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.2:45099 every 8 connection(s)
I20260807 02:31:09.156474 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/data/info.pb
I20260807 02:31:09.162878 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 15311
I20260807 02:31:09.163033 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal/instance
I20260807 02:31:09.163379 17620 external_mini_cluster.cc:1428] Setting key 8e8adab787418eafab35fe2bff1dbed7
I20260807 02:31:09.165473 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.53.3:0
--local_ip_for_outbound_sockets=127.17.53.3
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--builtin_ntp_servers=127.17.53.20:42461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 02:31:09.166962 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.158511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.2:54611 (local address 127.17.53.62:34087)
0807 02:31:09.158702 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.158706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.159840 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 02:31:09.161382 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.161387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.161456 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.161573 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:09.162558 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.163223 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.164551 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.164868 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.164997 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.166362 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:09.166367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:09.166370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:09.166387 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:09.166421 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:09.166430 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:09.166516 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.166722 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.166798 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:31:09.167893 15516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.158913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34087 (local address 127.17.53.2:54611)
0807 02:31:09.159661 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.159706 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.161139 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.161779 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.161786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.161872 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:09.162385 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.162399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.163369 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.163380 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.164389 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.164401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.165047 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.165052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.165251 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.166037 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:09.166039 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:09.166085 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:09.166164 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:09.166533 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:09.166553 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:09.166559 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.166706 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.167021 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260807 02:31:09.169420 15512 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34087
I20260807 02:31:09.169718 15512 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:09.170339 15512 heartbeater.cc:511] Master 127.17.53.62:34087 requested a full tablet report, sending...
I20260807 02:31:09.171568 14989 ts_manager.cc:194] Registered new tserver with Master: b3bee311cf92428b864a9ec9ddbdc54f (127.17.53.2:45099)
I20260807 02:31:09.172236 14989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.2:54611
W20260807 02:31:09.288897 15518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:09.289183 15518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:09.289224 15518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:09.293324 15518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:09.293381 15518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:09.293465 15518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.53.3
I20260807 02:31:09.298391 15518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:42461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.53.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.53.3
--webserver_port=0
--tserver_master_addrs=127.17.53.62:34087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.53.3
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:09.299664 15518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:09.301185 15518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:09.312160 15524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:09.312228 15518 server_base.cc:1034] running on GCE node
W20260807 02:31:09.312198 15527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:09.312167 15525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:09.312953 15518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:09.313689 15518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:09.314893 15518 hybrid_clock.cc:648] HybridClock initialized: now 1786069869314879 us; error 26 us; skew 500 ppm
I20260807 02:31:09.316962 15518 webserver.cc:460] Webserver started at http://127.17.53.3:44499/ using document root <none> and password file <none>
I20260807 02:31:09.317807 15518 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:09.317956 15518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:09.318230 15518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:09.320897 15518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data/instance:
uuid: "a14f25dc1a2243c4897f884bdd986b86"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "cdc8a953569336279c331bb4f5e4181a"
server_key_iv: "1be1f444ba10d7f75ebdc5483779ba97"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.321556 15518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance:
uuid: "a14f25dc1a2243c4897f884bdd986b86"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "cdc8a953569336279c331bb4f5e4181a"
server_key_iv: "1be1f444ba10d7f75ebdc5483779ba97"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.325665 15518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 02:31:09.328493 15534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.329947 15518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:09.330102 15518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal
uuid: "a14f25dc1a2243c4897f884bdd986b86"
format_stamp: "Formatted at 2026-08-07 02:31:09 on dist-test-slave-g170"
server_key: "cdc8a953569336279c331bb4f5e4181a"
server_key_iv: "1be1f444ba10d7f75ebdc5483779ba97"
server_key_version: "encryptionkey@0"
I20260807 02:31:09.330227 15518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:09.342437 15518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:09.343209 15518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:09.343421 15518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:09.344126 15518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 02:31:09.345196 15518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 02:31:09.345271 15518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.345348 15518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 02:31:09.345392 15518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:09.354724 15518 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.3:35117
I20260807 02:31:09.354749 15711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.3:35117 every 8 connection(s)
I20260807 02:31:09.355703 15518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/data/info.pb
I20260807 02:31:09.364184 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 15518
I20260807 02:31:09.364331 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal/instance
I20260807 02:31:09.364684 17620 external_mini_cluster.cc:1428] Setting key e7e283797cb91c0db619319edfce3230
I20260807 02:31:09.365960 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.357947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:36465 (local address 127.17.53.62:34087)
0807 02:31:09.358115 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.358119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.359288 (+ 1169us) server_negotiation.cc:407] Connection header received
0807 02:31:09.360748 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.360752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.360842 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.360955 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:09.361867 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.362410 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.363581 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.363817 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.363932 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.365332 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:09.365336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:09.365339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:09.365354 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:09.365381 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:09.365393 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:09.365492 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.365725 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.365796 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 02:31:09.367116 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.358388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:34087 (local address 127.17.53.3:36465)
0807 02:31:09.359122 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.359170 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.360520 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.361135 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.361144 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.361226 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:09.361716 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.361728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.362552 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.362555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.363449 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.363458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.363979 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.363983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.364156 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.364973 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 02:31:09.364975 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:09.365035 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 02:31:09.365117 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:09.365531 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:09.365542 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:09.365550 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.365702 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.366001 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":117,"threads_started":1}
I20260807 02:31:09.368287 15713 heartbeater.cc:347] Connected to a master server at 127.17.53.62:34087
I20260807 02:31:09.368547 15713 heartbeater.cc:464] Registering TS with master...
I20260807 02:31:09.369197 15713 heartbeater.cc:511] Master 127.17.53.62:34087 requested a full tablet report, sending...
I20260807 02:31:09.370476 14989 ts_manager.cc:194] Registered new tserver with Master: a14f25dc1a2243c4897f884bdd986b86 (127.17.53.3:35117)
I20260807 02:31:09.371068 14989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.53.3:36465
I20260807 02:31:09.379094 17620 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 02:31:09.384933 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.381251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.17.53.62:34087)
0807 02:31:09.381394 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.381397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.381574 (+ 177us) server_negotiation.cc:407] Connection header received
0807 02:31:09.381729 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.381731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.381771 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.381864 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:09.382087 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.382592 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.383465 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.383664 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.383739 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.384488 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:09.384491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:09.384493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:09.384504 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:09.384523 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:09.384530 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:09.384591 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.384739 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.384829 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 02:31:09.387941 14989 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 02:31:09.391053 14989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 02:31:09.407564 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.401412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.17.53.1:33461)
0807 02:31:09.402492 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.402506 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.402535 (+ 29us) server_negotiation.cc:407] Connection header received
0807 02:31:09.402644 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.402650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.402883 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.403473 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:09.403659 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.404382 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.405959 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.406534 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.406773 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.407035 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.407125 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.407353 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":290,"threads_started":1}
I20260807 02:31:09.407550 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.400711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33461 (local address 127.0.0.1:45220)
0807 02:31:09.401910 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.401966 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.402119 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.403122 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.403126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.403158 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:09.403282 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.403288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.404510 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.404517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.405824 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.405835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.406772 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.406777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.406920 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.406973 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.407095 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.407374 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"spinlock_wait_cycles":3328,"thread_start_us":102,"threads_started":1}
I20260807 02:31:09.408113 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.401083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:45099 (local address 127.0.0.1:60332)
0807 02:31:09.402028 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.402044 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.402151 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.403111 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.403118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.403166 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:09.403303 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.403313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.405116 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.405120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.406190 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.406202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.407634 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.407642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.407758 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.407785 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.407842 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.407909 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":61,"spinlock_wait_cycles":3968,"thread_start_us":119,"threads_started":1}
I20260807 02:31:09.409739 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.401083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.3:35117 (local address 127.0.0.1:58386)
0807 02:31:09.402554 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.402571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.402748 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.403373 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.403377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.403402 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:09.403488 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.403495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.405116 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.405120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.407517 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.407533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.409189 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.409194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.409306 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.409334 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.409455 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.409531 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1404,"mutex_wait_us":720,"thread_start_us":119,"threads_started":1}
I20260807 02:31:09.409678 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.402028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.17.53.2:45099)
0807 02:31:09.402491 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.402504 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.402533 (+ 29us) server_negotiation.cc:407] Connection header received
0807 02:31:09.402627 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.402636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.402913 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.403462 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:09.403617 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.404964 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.406333 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.407476 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.409184 (+ 1708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.409362 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.409416 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.409506 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260807 02:31:09.409812 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.401385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58386 (local address 127.17.53.3:35117)
0807 02:31:09.401790 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.401806 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.402672 (+ 866us) server_negotiation.cc:407] Connection header received
0807 02:31:09.402988 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.402998 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.403211 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.403396 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:09.403618 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.404960 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.407679 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.408702 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.408843 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.409059 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.409496 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.409626 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":177,"threads_started":1}
I20260807 02:31:09.411288 15190 tablet_service.cc:1467] Processing CreateTablet for tablet cfe37b05243643a08afb862f1e892d65 (DEFAULT_TABLE table=test-table [id=0aa36a5460694491a39581a122b272c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:09.412566 15190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe37b05243643a08afb862f1e892d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:09.412895 15616 tablet_service.cc:1467] Processing CreateTablet for tablet cfe37b05243643a08afb862f1e892d65 (DEFAULT_TABLE table=test-table [id=0aa36a5460694491a39581a122b272c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:09.412896 15413 tablet_service.cc:1467] Processing CreateTablet for tablet cfe37b05243643a08afb862f1e892d65 (DEFAULT_TABLE table=test-table [id=0aa36a5460694491a39581a122b272c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 02:31:09.414075 15616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe37b05243643a08afb862f1e892d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:09.414079 15413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe37b05243643a08afb862f1e892d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:09.419395 15735 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Bootstrap starting.
I20260807 02:31:09.421766 15735 tablet_bootstrap.cc:654] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:09.422202 15736 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Bootstrap starting.
I20260807 02:31:09.423178 15735 log.cc:824] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:09.424415 15736 tablet_bootstrap.cc:654] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:09.425135 15737 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Bootstrap starting.
I20260807 02:31:09.425292 15736 log.cc:824] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:09.425534 15735 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: No bootstrap required, opened a new log
I20260807 02:31:09.425789 15735 ts_tablet_manager.cc:1397] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260807 02:31:09.427361 15737 tablet_bootstrap.cc:654] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:09.427456 15736 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: No bootstrap required, opened a new log
I20260807 02:31:09.427647 15736 ts_tablet_manager.cc:1397] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 02:31:09.428246 15737 log.cc:824] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:09.430068 15735 raft_consensus.cc:348] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.430413 15735 raft_consensus.cc:374] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:09.430480 15735 raft_consensus.cc:729] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a55f409cea42238e37dd9666214dd5, State: Initialized, Role: FOLLOWER
I20260807 02:31:09.430732 15737 tablet_bootstrap.cc:492] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: No bootstrap required, opened a new log
I20260807 02:31:09.431142 15737 ts_tablet_manager.cc:1397] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 02:31:09.431097 15735 consensus_queue.cc:260] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.431879 15307 heartbeater.cc:503] Master 127.17.53.62:34087 was elected leader, sending a full tablet report...
I20260807 02:31:09.432030 15736 raft_consensus.cc:348] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.432279 15736 raft_consensus.cc:374] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:09.432281 15735 ts_tablet_manager.cc:1428] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260807 02:31:09.432339 15736 raft_consensus.cc:729] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3bee311cf92428b864a9ec9ddbdc54f, State: Initialized, Role: FOLLOWER
I20260807 02:31:09.432816 15736 consensus_queue.cc:260] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.433729 15512 heartbeater.cc:503] Master 127.17.53.62:34087 was elected leader, sending a full tablet report...
I20260807 02:31:09.433851 15736 ts_tablet_manager.cc:1428] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 02:31:09.436892 15737 raft_consensus.cc:348] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.437192 15737 raft_consensus.cc:374] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:09.437260 15737 raft_consensus.cc:729] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14f25dc1a2243c4897f884bdd986b86, State: Initialized, Role: FOLLOWER
I20260807 02:31:09.437618 15737 consensus_queue.cc:260] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.438239 15713 heartbeater.cc:503] Master 127.17.53.62:34087 was elected leader, sending a full tablet report...
I20260807 02:31:09.438512 15737 ts_tablet_manager.cc:1428] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 02:31:09.444983 15308 tablet.cc:2369] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:09.459024 15756 raft_consensus.cc:482] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 02:31:09.459218 15756 raft_consensus.cc:504] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.460362 15756 leader_election.cc:288] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3a55f409cea42238e37dd9666214dd5 (127.17.53.1:33461), b3bee311cf92428b864a9ec9ddbdc54f (127.17.53.2:45099)
I20260807 02:31:09.464653 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.460653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.2:45099 (local address 127.17.53.3:43945)
0807 02:31:09.460864 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.460879 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.460997 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.461289 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.461293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.461313 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:09.461442 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.461449 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.462505 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.462512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.463488 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.463498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.464282 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.464285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.464367 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.464396 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.464471 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.464530 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":3}
I20260807 02:31:09.464746 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.460858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:43945 (local address 127.17.53.2:45099)
0807 02:31:09.460998 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.461002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.461018 (+ 16us) server_negotiation.cc:407] Connection header received
0807 02:31:09.461107 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.461110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.461154 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.461257 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:09.461630 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.462389 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.463626 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.464125 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.464244 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.464313 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.464519 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.464602 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 02:31:09.465349 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.460653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33461 (local address 127.17.53.3:56205)
0807 02:31:09.461399 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.461411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:09.461495 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:09.461744 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:09.461747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:09.461766 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 02:31:09.461890 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.461896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.462975 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.462979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.463912 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:09.463927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.464971 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.464976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:09.465070 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.465094 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 02:31:09.465190 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 02:31:09.465243 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":145,"threads_started":1}
I20260807 02:31:09.465446 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.460847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.53.3:56205 (local address 127.17.53.1:33461)
0807 02:31:09.460999 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.461004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.461508 (+ 504us) server_negotiation.cc:407] Connection header received
0807 02:31:09.461584 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.461590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.461648 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.461712 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 02:31:09.462052 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.462800 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.464052 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.464669 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.464813 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.464883 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.465234 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.465326 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:31:09.465523 15449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe37b05243643a08afb862f1e892d65" candidate_uuid: "a14f25dc1a2243c4897f884bdd986b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" is_pre_election: true
I20260807 02:31:09.465895 15449 raft_consensus.cc:2454] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14f25dc1a2243c4897f884bdd986b86 in term 0.
I20260807 02:31:09.466059 15219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe37b05243643a08afb862f1e892d65" candidate_uuid: "a14f25dc1a2243c4897f884bdd986b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3a55f409cea42238e37dd9666214dd5" is_pre_election: true
I20260807 02:31:09.466346 15219 raft_consensus.cc:2454] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14f25dc1a2243c4897f884bdd986b86 in term 0.
I20260807 02:31:09.466516 15535 leader_election.cc:302] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a14f25dc1a2243c4897f884bdd986b86, b3bee311cf92428b864a9ec9ddbdc54f; no voters:
I20260807 02:31:09.466832 15756 raft_consensus.cc:2781] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 02:31:09.466930 15756 raft_consensus.cc:482] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 02:31:09.466980 15756 raft_consensus.cc:3037] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:09.468098 15756 raft_consensus.cc:504] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.468485 15756 leader_election.cc:288] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [CANDIDATE]: Term 1 election: Requested vote from peers f3a55f409cea42238e37dd9666214dd5 (127.17.53.1:33461), b3bee311cf92428b864a9ec9ddbdc54f (127.17.53.2:45099)
I20260807 02:31:09.468886 15219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe37b05243643a08afb862f1e892d65" candidate_uuid: "a14f25dc1a2243c4897f884bdd986b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3a55f409cea42238e37dd9666214dd5"
I20260807 02:31:09.468902 15449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe37b05243643a08afb862f1e892d65" candidate_uuid: "a14f25dc1a2243c4897f884bdd986b86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3bee311cf92428b864a9ec9ddbdc54f"
I20260807 02:31:09.469036 15219 raft_consensus.cc:3037] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:09.469048 15449 raft_consensus.cc:3037] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:09.470189 15449 raft_consensus.cc:2454] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14f25dc1a2243c4897f884bdd986b86 in term 1.
I20260807 02:31:09.470259 15219 raft_consensus.cc:2454] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14f25dc1a2243c4897f884bdd986b86 in term 1.
I20260807 02:31:09.470566 15535 leader_election.cc:302] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a14f25dc1a2243c4897f884bdd986b86, b3bee311cf92428b864a9ec9ddbdc54f; no voters:
I20260807 02:31:09.470784 15756 raft_consensus.cc:2781] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:09.470999 15756 raft_consensus.cc:686] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 1 LEADER]: Becoming Leader. State: Replica: a14f25dc1a2243c4897f884bdd986b86, State: Running, Role: LEADER
I20260807 02:31:09.471362 15756 consensus_queue.cc:237] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } }
I20260807 02:31:09.475167 14988 catalog_manager.cc:5495] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 reported cstate change: term changed from 0 to 1, leader changed from <none> to a14f25dc1a2243c4897f884bdd986b86 (127.17.53.3). New cstate: current_term: 1 leader_uuid: "a14f25dc1a2243c4897f884bdd986b86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a14f25dc1a2243c4897f884bdd986b86" member_type: VOTER last_known_addr { host: "127.17.53.3" port: 35117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 } health_report { overall_health: UNKNOWN } } }
I20260807 02:31:09.487247 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.483415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.17.53.3:35117)
0807 02:31:09.483563 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:09.483569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:09.483584 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:31:09.483795 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:09.483798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:09.483861 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:09.483983 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:09.484232 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.484923 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.485795 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:09.486016 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:09.486123 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:09.486448 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:09.486853 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:09.486983 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:09.487054 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 02:31:09.487104 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:31:09.494282 15219 raft_consensus.cc:1261] T cfe37b05243643a08afb862f1e892d65 P f3a55f409cea42238e37dd9666214dd5 [term 1 FOLLOWER]: Refusing update from remote peer a14f25dc1a2243c4897f884bdd986b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:09.494294 15449 raft_consensus.cc:1261] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 1 FOLLOWER]: Refusing update from remote peer a14f25dc1a2243c4897f884bdd986b86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:09.495061 15756 consensus_queue.cc:1035] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a55f409cea42238e37dd9666214dd5" member_type: VOTER last_known_addr { host: "127.17.53.1" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:09.495267 15758 consensus_queue.cc:1035] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3bee311cf92428b864a9ec9ddbdc54f" member_type: VOTER last_known_addr { host: "127.17.53.2" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:09.502807 15764 mvcc.cc:205] Tried to move back new op lower bound from 7315742185438679040 to 7315742185358221312. Current Snapshot: MvccSnapshot[applied={T|T < 7315742185438679040}]
I20260807 02:31:09.504106 15767 mvcc.cc:205] Tried to move back new op lower bound from 7315742185438679040 to 7315742185358221312. Current Snapshot: MvccSnapshot[applied={T|T < 7315742185438679040}]
I20260807 02:31:09.511951 14988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:09.512130 14988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 02:31:09.514642 14988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a8e8c5f57024cadb02d0a107db16142: Sending DeleteTablet for 3 replicas of tablet cfe37b05243643a08afb862f1e892d65
I20260807 02:31:09.515424 15190 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe37b05243643a08afb862f1e892d65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:09 UTC) from {username='slave'} at 127.0.0.1:45220
I20260807 02:31:09.515479 15413 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe37b05243643a08afb862f1e892d65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:09 UTC) from {username='slave'} at 127.0.0.1:60332
I20260807 02:31:09.515483 15616 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe37b05243643a08afb862f1e892d65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 02:31:09 UTC) from {username='slave'} at 127.0.0.1:58386
I20260807 02:31:09.515969 15783 tablet_replica.cc:330] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: stopping tablet replica
I20260807 02:31:09.515988 15785 tablet_replica.cc:330] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: stopping tablet replica
I20260807 02:31:09.516302 15785 raft_consensus.cc:2229] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 1 LEADER]: Raft consensus shutting down.
I20260807 02:31:09.516340 15783 raft_consensus.cc:2229] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 02:31:09.516712 15785 raft_consensus.cc:2258] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:09.516891 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 15078
I20260807 02:31:09.516875 15783 raft_consensus.cc:2258] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 02:31:09.521281 15785 ts_tablet_manager.cc:1905] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:09.521735 15783 ts_tablet_manager.cc:1905] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 02:31:09.524833 15783 ts_tablet_manager.cc:1918] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:09.525059 15783 log.cc:1196] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-1/wal/wals/cfe37b05243643a08afb862f1e892d65
I20260807 02:31:09.525256 15785 ts_tablet_manager.cc:1918] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 02:31:09.525365 15785 log.cc:1196] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Deleting WAL directory at /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786069602236550-17620-0/minicluster-data/ts-2/wal/wals/cfe37b05243643a08afb862f1e892d65
I20260807 02:31:09.525642 15783 ts_tablet_manager.cc:1939] T cfe37b05243643a08afb862f1e892d65 P b3bee311cf92428b864a9ec9ddbdc54f: Deleting consensus metadata
I20260807 02:31:09.525731 15785 ts_tablet_manager.cc:1939] T cfe37b05243643a08afb862f1e892d65 P a14f25dc1a2243c4897f884bdd986b86: Deleting consensus metadata
I20260807 02:31:09.527086 14966 catalog_manager.cc:4841] TS b3bee311cf92428b864a9ec9ddbdc54f (127.17.53.2:45099): tablet cfe37b05243643a08afb862f1e892d65 (table test-table [id=0aa36a5460694491a39581a122b272c3]) successfully deleted
I20260807 02:31:09.527086 14969 catalog_manager.cc:4841] TS a14f25dc1a2243c4897f884bdd986b86 (127.17.53.3:35117): tablet cfe37b05243643a08afb862f1e892d65 (table test-table [id=0aa36a5460694491a39581a122b272c3]) successfully deleted
W20260807 02:31:09.533636 14967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.53.1:33461 (error 108)
I20260807 02:31:09.533769 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 15311
I20260807 02:31:09.537176 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:09.536719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.1:33461 (local address 127.0.0.1:45222)
0807 02:31:09.536999 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:09.537074 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.1:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
W20260807 02:31:09.537348 14967 catalog_manager.cc:4584] TS f3a55f409cea42238e37dd9666214dd5 (127.17.53.1:33461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfe37b05243643a08afb862f1e892d65: Network error: Client connection negotiation failed: client connection to 127.17.53.1:33461: connect: Connection refused (error 111)
I20260807 02:31:09.545001 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 15518
I20260807 02:31:09.559471 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 14941
2026-08-07T02:31:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1117 ms)
[----------] 6 tests from EncryptionPolicyTest (11819 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:09 dist-test-slave-g170 krb5kdc[15790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:09 dist-test-slave-g170 krb5kdc[15790](info): set up 2 sockets
Aug 07 02:31:09 dist-test-slave-g170 krb5kdc[15790](info): commencing operation
krb5kdc: starting...
W20260807 02:31:10.803519 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:10 dist-test-slave-g170 krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:10.937494 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:11.061652 15813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:11.061925 15813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:11.061967 15813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:11.065856 15813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:11.065907 15813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:11.065927 15813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:11.065941 15813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:11.065956 15813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:11.071087 15813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:11.072052 15813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:11.073102 15813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:11.081813 15824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:11.081804 15821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:11.081813 15820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:11.082041 15813 server_base.cc:1034] running on GCE node
I20260807 02:31:11.082679 15813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:11.083818 15813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:11.084985 15813 hybrid_clock.cc:648] HybridClock initialized: now 1786069871084956 us; error 41 us; skew 500 ppm
Aug 07 02:31:11 dist-test-slave-g170 krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:11.093835 15813 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:11.094790 15813 webserver.cc:460] Webserver started at http://127.0.0.1:33757/ using document root <none> and password file <none>
I20260807 02:31:11.095379 15813 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:11.095444 15813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:11.095626 15813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:11.097856 15813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "90dc20e8f7d74b8eab45f9c3495a8d57"
format_stamp: "Formatted at 2026-08-07 02:31:11 on dist-test-slave-g170"
server_key: "47c57057a5390538ecdb3b5237234ea6"
server_key_iv: "6fd08d99854e212b6743423d020c318c"
server_key_version: "encryptionkey@0"
I20260807 02:31:11.098436 15813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "90dc20e8f7d74b8eab45f9c3495a8d57"
format_stamp: "Formatted at 2026-08-07 02:31:11 on dist-test-slave-g170"
server_key: "47c57057a5390538ecdb3b5237234ea6"
server_key_iv: "6fd08d99854e212b6743423d020c318c"
server_key_version: "encryptionkey@0"
I20260807 02:31:11.102370 15813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:11.104950 15831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:11.106235 15813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:11.106402 15813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "90dc20e8f7d74b8eab45f9c3495a8d57"
format_stamp: "Formatted at 2026-08-07 02:31:11 on dist-test-slave-g170"
server_key: "47c57057a5390538ecdb3b5237234ea6"
server_key_iv: "6fd08d99854e212b6743423d020c318c"
server_key_version: "encryptionkey@0"
I20260807 02:31:11.106509 15813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:11.126364 15813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:11.130018 15813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:11.130245 15813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:11.138479 15813 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33759
I20260807 02:31:11.138511 15910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33759 every 8 connection(s)
I20260807 02:31:11.139520 15813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:11.142482 15911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:11.145771 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 15813
I20260807 02:31:11.145998 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:11.146332 17620 external_mini_cluster.cc:1428] Setting key 6def5a7d8f132f12c6f111781d09648c
I20260807 02:31:11.149896 15911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: Bootstrap starting.
I20260807 02:31:11.152575 15911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:11.153522 15911 log.cc:824] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:11.156010 15911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: No bootstrap required, opened a new log
Aug 07 02:31:11 dist-test-slave-g170 krb5kdc[15790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 02:31:11.159739 15911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260807 02:31:11.160058 15911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:11.160127 15911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90dc20e8f7d74b8eab45f9c3495a8d57, State: Initialized, Role: FOLLOWER
I20260807 02:31:11.160708 15911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260807 02:31:11.160921 15911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:11.161002 15911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:11.161134 15911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:11.162339 15911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260807 02:31:11.162714 15911 leader_election.cc:302] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90dc20e8f7d74b8eab45f9c3495a8d57; no voters:
I20260807 02:31:11.163086 15911 leader_election.cc:288] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:11.163228 15919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:11.163477 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:11.147951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46983 (local address 127.0.0.1:33759)
0807 02:31:11.148426 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:11.148455 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:11.148496 (+ 41us) server_negotiation.cc:407] Connection header received
0807 02:31:11.149289 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:11.149319 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:11.149718 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:11.150059 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:11.150941 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:11.152017 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:11.153251 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:11.153874 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:11.154095 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:11.157093 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:11.157099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:11.157119 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:11.157155 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:11.159745 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:11.160377 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:11.160383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:11.160388 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:11.160511 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:11.160926 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:11.160930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:11.160932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:11.161523 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:11.161738 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:11.162043 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 02:31:11.162341 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":135,"threads_started":1}
I20260807 02:31:11.163501 15919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [term 1 LEADER]: Becoming Leader. State: Replica: 90dc20e8f7d74b8eab45f9c3495a8d57, State: Running, Role: LEADER
I20260807 02:31:11.163856 15919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } }
I20260807 02:31:11.164418 15911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:11.166558 15923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } } }
I20260807 02:31:11.166672 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90dc20e8f7d74b8eab45f9c3495a8d57. Latest consensus state: current_term: 1 leader_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dc20e8f7d74b8eab45f9c3495a8d57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33759 } } }
I20260807 02:31:11.166812 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:11.166841 15923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:11.167547 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 02:31:11.170442 15944 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:11.170528 15944 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:11.170603 15945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:11.171327 15945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:11.178426 15945 catalog_manager.cc:1347] Generated new cluster ID: be953bdcd642452e95219870adb61bcf
I20260807 02:31:11.178498 15945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:11.179471 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:11.169787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39533 (local address 127.0.0.1:33759)
0807 02:31:11.169923 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:11.169928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:11.170224 (+ 296us) server_negotiation.cc:407] Connection header received
0807 02:31:11.170377 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:11.170381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:11.170437 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:11.170531 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:11.171345 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:11.172185 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:11.173344 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:11.173583 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:11.173706 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:11.175152 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:11.175160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:11.175163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:11.175187 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:11.177036 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:11.177595 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:11.177599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:11.177600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:11.177716 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:11.178148 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:11.178151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:11.178153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:11.178389 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:11.178681 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:11.179129 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 02:31:11.179258 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:31:11.196210 15945 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:11.197386 15945 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:11.202692 15945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90dc20e8f7d74b8eab45f9c3495a8d57: Generated new TSK 0
I20260807 02:31:11.203442 15945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:11.210264 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 15813
W20260807 02:31:11.220422 15787 connection.cc:535] client connection to 127.0.0.1:33759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33759): Connection reset by peer (error 104)
2026-08-07T02:31:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:11 dist-test-slave-g170 krb5kdc[15964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:11 dist-test-slave-g170 krb5kdc[15964](info): set up 2 sockets
Aug 07 02:31:11 dist-test-slave-g170 krb5kdc[15964](info): commencing operation
krb5kdc: starting...
W20260807 02:31:12.486541 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:12 dist-test-slave-g170 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:12.628145 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:12.746441 15982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:12.746733 15982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:12.746780 15982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:12.751044 15982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:12.751121 15982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:12.751142 15982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:12.751159 15982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:12.751174 15982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:12.756824 15982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:12.758045 15982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:12.759228 15982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:12.769007 15998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:12.769007 15993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:12.769093 15982 server_base.cc:1034] running on GCE node
W20260807 02:31:12.768980 15994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:12.769986 15982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:12.771636 15982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:12.772872 15982 hybrid_clock.cc:648] HybridClock initialized: now 1786069872772826 us; error 67 us; skew 500 ppm
Aug 07 02:31:12 dist-test-slave-g170 krb5kdc[15964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:12.775988 15982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:12.777011 15982 webserver.cc:460] Webserver started at http://127.0.0.1:42645/ using document root <none> and password file <none>
I20260807 02:31:12.777562 15982 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:12.777622 15982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:12.777806 15982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:12.780031 15982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d2fed26ceb64471da2f4524216f976ef"
format_stamp: "Formatted at 2026-08-07 02:31:12 on dist-test-slave-g170"
server_key: "f0bbc323b0104e650e8494f5593d16d3"
server_key_iv: "7cf97c55c98b1ca20bc9ca36538a80b6"
server_key_version: "encryptionkey@0"
I20260807 02:31:12.780676 15982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d2fed26ceb64471da2f4524216f976ef"
format_stamp: "Formatted at 2026-08-07 02:31:12 on dist-test-slave-g170"
server_key: "f0bbc323b0104e650e8494f5593d16d3"
server_key_iv: "7cf97c55c98b1ca20bc9ca36538a80b6"
server_key_version: "encryptionkey@0"
I20260807 02:31:12.784862 15982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:31:12.787389 16011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:12.788820 15982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:12.788983 15982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d2fed26ceb64471da2f4524216f976ef"
format_stamp: "Formatted at 2026-08-07 02:31:12 on dist-test-slave-g170"
server_key: "f0bbc323b0104e650e8494f5593d16d3"
server_key_iv: "7cf97c55c98b1ca20bc9ca36538a80b6"
server_key_version: "encryptionkey@0"
I20260807 02:31:12.789089 15982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:12.810397 15982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:12.813666 15982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:12.813902 15982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:12.823118 15982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42477
I20260807 02:31:12.823139 16080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42477 every 8 connection(s)
I20260807 02:31:12.824150 15982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:12.826001 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 15982
I20260807 02:31:12.826149 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:12.826484 17620 external_mini_cluster.cc:1428] Setting key da91e9099a3a644f24aebedf73173cf9
I20260807 02:31:12.827476 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:12.833289 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: Bootstrap starting.
I20260807 02:31:12.835593 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:12.827919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39213 (local address 127.0.0.1:42477)
0807 02:31:12.828416 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:12.828432 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:12.828470 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:31:12.829195 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:12.829213 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:12.829584 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:12.829925 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:12.830329 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:12.831349 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:12.832243 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:12.832718 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:12.832976 (+ 258us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:12.833918 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:12.833923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:12.833936 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:12.834323 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:12.834377 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:12.834400 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:12.834547 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:12.834745 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 02:31:12.834887 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":150,"threads_started":1}
I20260807 02:31:12.835621 16081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:12.836481 16081 log.cc:824] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:12.838719 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: No bootstrap required, opened a new log
I20260807 02:31:12.839092 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:12.842890 16081 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 02:31:12.843169 16081 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:12.843225 16081 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2fed26ceb64471da2f4524216f976ef, State: Initialized, Role: FOLLOWER
I20260807 02:31:12.843710 16081 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 02:31:12.843935 16081 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:12.844036 16081 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:12.844174 16081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:12.845532 16081 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 02:31:12.845942 16081 leader_election.cc:302] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2fed26ceb64471da2f4524216f976ef; no voters:
I20260807 02:31:12.846076 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:12.841236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51061 (local address 127.0.0.1:42477)
0807 02:31:12.841369 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:12.841373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:12.841600 (+ 227us) server_negotiation.cc:407] Connection header received
0807 02:31:12.841710 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:12.841712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:12.841749 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:12.841815 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:12.842140 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:12.842679 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:12.843847 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:12.844357 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:12.844462 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:12.845392 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:12.845402 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:12.845404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:12.845418 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:12.845443 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:12.845454 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:12.845526 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:12.845811 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 02:31:12.845856 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:31:12.846297 16081 leader_election.cc:288] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:12.846439 16095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:12.846733 16095 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [term 1 LEADER]: Becoming Leader. State: Replica: d2fed26ceb64471da2f4524216f976ef, State: Running, Role: LEADER
I20260807 02:31:12.847417 16095 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } }
I20260807 02:31:12.847724 16081 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:12.849651 16097 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2fed26ceb64471da2f4524216f976ef. Latest consensus state: current_term: 1 leader_uuid: "d2fed26ceb64471da2f4524216f976ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260807 02:31:12.849804 16097 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:12.850203 16098 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2fed26ceb64471da2f4524216f976ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fed26ceb64471da2f4524216f976ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42477 } } }
I20260807 02:31:12.850306 16098 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:12.853015 16121 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:12.853099 16121 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:12.853155 16122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:12.854031 16122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:12.859949 16122 catalog_manager.cc:1347] Generated new cluster ID: ee3d65a1cce04c5fb7833c807e6f6ad6
I20260807 02:31:12.860033 16122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:12.868422 16122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:12.869661 16122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:12.883998 16122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2fed26ceb64471da2f4524216f976ef: Generated new TSK 0
I20260807 02:31:12.884864 16122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:12.889264 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 15982
W20260807 02:31:12.903285 15962 connection.cc:535] client connection to 127.0.0.1:42477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42477): Connection reset by peer (error 104)
2026-08-07T02:31:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:12 dist-test-slave-g170 krb5kdc[16130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:12 dist-test-slave-g170 krb5kdc[16130](info): set up 2 sockets
Aug 07 02:31:12 dist-test-slave-g170 krb5kdc[16130](info): commencing operation
krb5kdc: starting...
W20260807 02:31:14.141120 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:14 dist-test-slave-g170 krb5kdc[16130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:14.276927 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:14.393839 16147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:14.394102 16147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:14.394145 16147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:14.398196 16147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:14.398257 16147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:14.398283 16147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:14.398299 16147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:14.398353 16147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:14.403659 16147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46645
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:14.404680 16147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:14.405763 16147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:14.414779 16157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:14.414878 16147 server_base.cc:1034] running on GCE node
W20260807 02:31:14.414794 16152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:14.414825 16153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:14.415545 16147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:14.416671 16147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:14.417856 16147 hybrid_clock.cc:648] HybridClock initialized: now 1786069874417824 us; error 49 us; skew 500 ppm
Aug 07 02:31:14 dist-test-slave-g170 krb5kdc[16130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:14.420660 16147 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:14.421577 16147 webserver.cc:460] Webserver started at http://127.0.0.1:41455/ using document root <none> and password file <none>
I20260807 02:31:14.422092 16147 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:14.422148 16147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:14.422318 16147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:14.424296 16147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "4ba9fac915b042e88718772fb7d4a4b8"
format_stamp: "Formatted at 2026-08-07 02:31:14 on dist-test-slave-g170"
server_key: "72b0748feb7c68d7f0e34f25b548998e"
server_key_iv: "967f7434efa2b4840e2d7392f3dff8d1"
server_key_version: "encryptionkey@0"
I20260807 02:31:14.424873 16147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "4ba9fac915b042e88718772fb7d4a4b8"
format_stamp: "Formatted at 2026-08-07 02:31:14 on dist-test-slave-g170"
server_key: "72b0748feb7c68d7f0e34f25b548998e"
server_key_iv: "967f7434efa2b4840e2d7392f3dff8d1"
server_key_version: "encryptionkey@0"
I20260807 02:31:14.428500 16147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:14.430999 16172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:14.432273 16147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:31:14.432371 16147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "4ba9fac915b042e88718772fb7d4a4b8"
format_stamp: "Formatted at 2026-08-07 02:31:14 on dist-test-slave-g170"
server_key: "72b0748feb7c68d7f0e34f25b548998e"
server_key_iv: "967f7434efa2b4840e2d7392f3dff8d1"
server_key_version: "encryptionkey@0"
I20260807 02:31:14.432448 16147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:14.436981 16147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:14.440124 16147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:14.440266 16147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:14.447769 16147 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46645
I20260807 02:31:14.447789 16249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46645 every 8 connection(s)
I20260807 02:31:14.448722 16147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:14.451866 16250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:14.453960 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 16147
I20260807 02:31:14.454115 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:14.454420 17620 external_mini_cluster.cc:1428] Setting key 589a5ea5c15642fddac9650f9f62b3a4
I20260807 02:31:14.459407 16250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8: Bootstrap starting.
I20260807 02:31:14.460289 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:14.455934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48745 (local address 127.0.0.1:46645)
0807 02:31:14.456395 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:14.456416 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:14.456461 (+ 45us) server_negotiation.cc:407] Connection header received
0807 02:31:14.457347 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:14.457372 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:14.457754 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:14.458140 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:14.458899 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:14.458904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:14.458920 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:14.458932 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:14.458996 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:14.459023 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:14.459164 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:14.459245 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 02:31:14.459399 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":118,"threads_started":1}
I20260807 02:31:14.462138 16250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:14.463065 16250 log.cc:824] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:14.463456 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:14.465476 16250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8: No bootstrap required, opened a new log
I20260807 02:31:14.467502 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:14.465760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55627 (local address 127.0.0.1:46645)
0807 02:31:14.465895 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:14.465900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:14.466108 (+ 208us) server_negotiation.cc:407] Connection header received
0807 02:31:14.466249 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:14.466252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:14.466299 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:14.466382 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:14.467106 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:14.467110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:14.467113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:14.467125 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:14.467152 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:14.467171 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:14.467271 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:14.467339 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 02:31:14.467381 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 02:31:14.469846 16250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } }
I20260807 02:31:14.470090 16250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:14.470194 16250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba9fac915b042e88718772fb7d4a4b8, State: Initialized, Role: FOLLOWER
I20260807 02:31:14.470697 16250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } }
I20260807 02:31:14.470857 16250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:14.470961 16250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:14.471096 16250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:14.472124 16250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } }
I20260807 02:31:14.472517 16250 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba9fac915b042e88718772fb7d4a4b8; no voters:
I20260807 02:31:14.472949 16250 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:14.473066 16267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:14.473316 16267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ba9fac915b042e88718772fb7d4a4b8, State: Running, Role: LEADER
I20260807 02:31:14.473659 16267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } }
I20260807 02:31:14.474375 16250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:14.475468 16271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba9fac915b042e88718772fb7d4a4b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } } }
I20260807 02:31:14.475602 16271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:14.475857 16272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba9fac915b042e88718772fb7d4a4b8. Latest consensus state: current_term: 1 leader_uuid: "4ba9fac915b042e88718772fb7d4a4b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9fac915b042e88718772fb7d4a4b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46645 } } }
I20260807 02:31:14.475952 16272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:14.476104 16286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:14.479120 16286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:14.484867 16286 catalog_manager.cc:1347] Generated new cluster ID: c673e7987d22403a951cf3d5b35904a4
I20260807 02:31:14.484937 16286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:14.514416 16286 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:14.514804 16286 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:14.519635 16286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ba9fac915b042e88718772fb7d4a4b8: Generated new TSK 0
I20260807 02:31:14.520498 16286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:14.536473 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 16147
2026-08-07T02:31:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:14 dist-test-slave-g170 krb5kdc[16303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:14 dist-test-slave-g170 krb5kdc[16303](info): set up 2 sockets
Aug 07 02:31:14 dist-test-slave-g170 krb5kdc[16303](info): commencing operation
krb5kdc: starting...
W20260807 02:31:15.821174 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:15 dist-test-slave-g170 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:15.955955 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:16.080260 16328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:16.080575 16328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:16.080673 16328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:16.084928 16328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:16.085016 16328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:16.085050 16328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:16.085088 16328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:16.085122 16328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:16.090701 16328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:16.091898 16328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:16.093051 16328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:16.101977 16337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:16.102010 16335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:16.101982 16334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:16.102304 16328 server_base.cc:1034] running on GCE node
I20260807 02:31:16.102938 16328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:16.104162 16328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:16.105360 16328 hybrid_clock.cc:648] HybridClock initialized: now 1786069876105347 us; error 31 us; skew 500 ppm
Aug 07 02:31:16 dist-test-slave-g170 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:16.111584 16328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:16.112605 16328 webserver.cc:460] Webserver started at http://127.0.0.1:40977/ using document root <none> and password file <none>
I20260807 02:31:16.113252 16328 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:16.113348 16328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:16.113592 16328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:16.115651 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "7eddb6489b894bc7bddb089c8095b571"
format_stamp: "Formatted at 2026-08-07 02:31:16 on dist-test-slave-g170"
server_key: "a6a26a27f8feba92fc034dc0793d2c2e"
server_key_iv: "fefc94b1301440b3b0e625f8732dfd9b"
server_key_version: "encryptionkey@0"
I20260807 02:31:16.116214 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "7eddb6489b894bc7bddb089c8095b571"
format_stamp: "Formatted at 2026-08-07 02:31:16 on dist-test-slave-g170"
server_key: "a6a26a27f8feba92fc034dc0793d2c2e"
server_key_iv: "fefc94b1301440b3b0e625f8732dfd9b"
server_key_version: "encryptionkey@0"
I20260807 02:31:16.120023 16328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:16.122578 16347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:16.123857 16328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 02:31:16.124023 16328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "7eddb6489b894bc7bddb089c8095b571"
format_stamp: "Formatted at 2026-08-07 02:31:16 on dist-test-slave-g170"
server_key: "a6a26a27f8feba92fc034dc0793d2c2e"
server_key_iv: "fefc94b1301440b3b0e625f8732dfd9b"
server_key_version: "encryptionkey@0"
I20260807 02:31:16.124177 16328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:16.146036 16328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:16.149708 16328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:16.149950 16328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:16.158027 16328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45421
I20260807 02:31:16.158115 16431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45421 every 8 connection(s)
I20260807 02:31:16.159147 16328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:16.162488 16434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:16.164345 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 16328
I20260807 02:31:16.164531 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:16.164985 17620 external_mini_cluster.cc:1428] Setting key 8c88400dd2d490b8d62967ea53170604
I20260807 02:31:16.170094 16434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571: Bootstrap starting.
I20260807 02:31:16.172833 16434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:16.173956 16434 log.cc:824] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:31:16 dist-test-slave-g170 krb5kdc[16303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 02:31:16.176565 16434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571: No bootstrap required, opened a new log
I20260807 02:31:16.180670 16434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260807 02:31:16.181059 16434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:16.181149 16434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eddb6489b894bc7bddb089c8095b571, State: Initialized, Role: FOLLOWER
I20260807 02:31:16.181651 16434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260807 02:31:16.181797 16434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:16.181903 16434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:16.182036 16434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:16.182210 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:16.166690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56821 (local address 127.0.0.1:45421)
0807 02:31:16.167228 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:16.167245 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:16.167291 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:31:16.168045 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:16.168067 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:16.168438 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:16.168889 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:16.169839 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:16.171049 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:16.172023 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:16.172544 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:16.172739 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:16.176754 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:16.176803 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:16.176828 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:16.176857 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:16.178735 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:16.179225 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:16.179231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:16.179235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:16.179350 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:16.179674 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:16.179677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:16.179678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:16.180127 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:16.180374 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:16.180710 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 02:31:16.181188 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":176,"threads_started":1}
I20260807 02:31:16.183248 16434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260807 02:31:16.183679 16434 leader_election.cc:302] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eddb6489b894bc7bddb089c8095b571; no voters:
I20260807 02:31:16.184145 16434 leader_election.cc:288] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:16.184283 16442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:16.184545 16442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [term 1 LEADER]: Becoming Leader. State: Replica: 7eddb6489b894bc7bddb089c8095b571, State: Running, Role: LEADER
I20260807 02:31:16.184981 16442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } }
I20260807 02:31:16.185760 16434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:16.185935 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:16.186259 16442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eddb6489b894bc7bddb089c8095b571. Latest consensus state: current_term: 1 leader_uuid: "7eddb6489b894bc7bddb089c8095b571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } } }
I20260807 02:31:16.186376 16442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:16.186790 16448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:16.186985 16444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eddb6489b894bc7bddb089c8095b571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eddb6489b894bc7bddb089c8095b571" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45421 } } }
I20260807 02:31:16.187196 16444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:16.190748 16448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:16.198850 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:16.188451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51989 (local address 127.0.0.1:45421)
0807 02:31:16.188560 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:16.188564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:16.189337 (+ 773us) server_negotiation.cc:407] Connection header received
0807 02:31:16.189533 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:16.189536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:16.189585 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:16.189637 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:16.190597 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:16.191477 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:16.192914 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:16.193227 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:16.193353 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:16.194791 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:16.194795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:16.194800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:16.194828 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:16.196472 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:16.197134 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:16.197137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:16.197139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:16.197238 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:16.197579 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:16.197582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:16.197584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:16.197829 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:16.198542 (+ 713us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:16.198623 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 02:31:16.198724 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 02:31:16.199090 16448 catalog_manager.cc:1347] Generated new cluster ID: 9117e072ae73448e96b75aaeabfa49e9
I20260807 02:31:16.199138 16448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:16.222388 16448 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:16.223778 16448 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:16.230618 16448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eddb6489b894bc7bddb089c8095b571: Generated new TSK 0
I20260807 02:31:16.231702 16448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:16.246300 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 16328
2026-08-07T02:31:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1702 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:16 dist-test-slave-g170 krb5kdc[16480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:16 dist-test-slave-g170 krb5kdc[16480](info): set up 2 sockets
Aug 07 02:31:16 dist-test-slave-g170 krb5kdc[16480](info): commencing operation
krb5kdc: starting...
W20260807 02:31:17.578606 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:17 dist-test-slave-g170 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:17.714015 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:17.831472 16496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:17.831812 16496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:17.831887 16496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:17.836054 16496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:17.836159 16496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:17.836203 16496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:17.836269 16496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:17.836310 16496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:17.841846 16496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:17.843001 16496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:17.844089 16496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:17.853276 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:17.853288 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:17.853288 16509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:17.853775 16496 server_base.cc:1034] running on GCE node
I20260807 02:31:17.854450 16496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:17.855672 16496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:17.856892 16496 hybrid_clock.cc:648] HybridClock initialized: now 1786069877856864 us; error 40 us; skew 500 ppm
Aug 07 02:31:17 dist-test-slave-g170 krb5kdc[16480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:17.860262 16496 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:17.861446 16496 webserver.cc:460] Webserver started at http://127.0.0.1:44343/ using document root <none> and password file <none>
I20260807 02:31:17.862134 16496 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:17.862208 16496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:17.862452 16496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:17.864876 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "b92c1b5d1005494ca39694035a9c73df"
format_stamp: "Formatted at 2026-08-07 02:31:17 on dist-test-slave-g170"
server_key: "b1337f031751ca77521f352e29db6f1c"
server_key_iv: "aafc0c7ddb4bf5c873ff5647b110b61c"
server_key_version: "encryptionkey@0"
I20260807 02:31:17.865839 16496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "b92c1b5d1005494ca39694035a9c73df"
format_stamp: "Formatted at 2026-08-07 02:31:17 on dist-test-slave-g170"
server_key: "b1337f031751ca77521f352e29db6f1c"
server_key_iv: "aafc0c7ddb4bf5c873ff5647b110b61c"
server_key_version: "encryptionkey@0"
I20260807 02:31:17.870306 16496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:31:17.873456 16519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:17.875030 16496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 02:31:17.875303 16496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "b92c1b5d1005494ca39694035a9c73df"
format_stamp: "Formatted at 2026-08-07 02:31:17 on dist-test-slave-g170"
server_key: "b1337f031751ca77521f352e29db6f1c"
server_key_iv: "aafc0c7ddb4bf5c873ff5647b110b61c"
server_key_version: "encryptionkey@0"
I20260807 02:31:17.875429 16496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:17.903745 16496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:17.907407 16496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:17.907634 16496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:17.915397 16496 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34001
I20260807 02:31:17.915490 16605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34001 every 8 connection(s)
I20260807 02:31:17.916474 16496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:17.919516 16606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:17.922060 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 16496
I20260807 02:31:17.922179 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:17.922501 17620 external_mini_cluster.cc:1428] Setting key 9b1955293d7be05d78351f0403f14536
I20260807 02:31:17.926031 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df: Bootstrap starting.
I20260807 02:31:17.928826 16606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:17.929736 16606 log.cc:824] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:17.932032 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df: No bootstrap required, opened a new log
I20260807 02:31:17.932307 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:17.924227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53865 (local address 127.0.0.1:34001)
0807 02:31:17.924707 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:17.924724 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:17.924807 (+ 83us) server_negotiation.cc:407] Connection header received
0807 02:31:17.925507 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:17.925526 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:17.925895 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:17.926242 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:17.926587 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:17.927641 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:17.928924 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:17.929441 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:17.929577 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:17.930445 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:17.930454 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:17.930475 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:17.930930 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:17.930986 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:17.931010 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:17.931090 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:17.931274 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 02:31:17.931512 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":168,"threads_started":1}
I20260807 02:31:17.935326 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:17.935956 16606 raft_consensus.cc:348] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260807 02:31:17.936173 16606 raft_consensus.cc:374] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:17.936259 16606 raft_consensus.cc:729] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92c1b5d1005494ca39694035a9c73df, State: Initialized, Role: FOLLOWER
I20260807 02:31:17.936740 16606 consensus_queue.cc:260] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260807 02:31:17.936923 16606 raft_consensus.cc:388] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:17.937006 16606 raft_consensus.cc:482] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:17.937121 16606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:17.938441 16606 raft_consensus.cc:504] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260807 02:31:17.938812 16606 leader_election.cc:302] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b92c1b5d1005494ca39694035a9c73df; no voters:
I20260807 02:31:17.939179 16606 leader_election.cc:288] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:17.939257 16621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:17.939479 16621 raft_consensus.cc:686] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [term 1 LEADER]: Becoming Leader. State: Replica: b92c1b5d1005494ca39694035a9c73df, State: Running, Role: LEADER
I20260807 02:31:17.939792 16621 consensus_queue.cc:237] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260807 02:31:17.940541 16606 sys_catalog.cc:564] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:17.941061 16621 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92c1b5d1005494ca39694035a9c73df. Latest consensus state: current_term: 1 leader_uuid: "b92c1b5d1005494ca39694035a9c73df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260807 02:31:17.941215 16621 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:17.941543 16622 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b92c1b5d1005494ca39694035a9c73df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92c1b5d1005494ca39694035a9c73df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260807 02:31:17.941610 16625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:17.941686 16622 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:17.942555 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:17.937535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40619 (local address 127.0.0.1:34001)
0807 02:31:17.937663 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:17.937667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:17.937860 (+ 193us) server_negotiation.cc:407] Connection header received
0807 02:31:17.937988 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:17.937990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:17.938027 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:17.938090 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:17.938449 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:17.939083 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:17.940169 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:17.940396 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:17.940510 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:17.942037 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:17.942041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:17.942044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:17.942056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:17.942085 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:17.942097 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:17.942143 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:17.942325 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 02:31:17.942395 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":48}
I20260807 02:31:17.945161 16625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:17.951298 16625 catalog_manager.cc:1347] Generated new cluster ID: a05c0b4e551448d8831d7d5a178050ce
I20260807 02:31:17.951370 16625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:17.969183 16625 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:17.970630 16625 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:17.978705 16625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b92c1b5d1005494ca39694035a9c73df: Generated new TSK 0
I20260807 02:31:17.979851 16625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:17.993078 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 16496
2026-08-07T02:31:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1749 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:18 dist-test-slave-g170 krb5kdc[16651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:18 dist-test-slave-g170 krb5kdc[16651](info): set up 2 sockets
Aug 07 02:31:18 dist-test-slave-g170 krb5kdc[16651](info): commencing operation
krb5kdc: starting...
W20260807 02:31:19.263677 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:19 dist-test-slave-g170 krb5kdc[16651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:19.404712 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:19.518965 16667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:19.519217 16667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:19.519256 16667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:19.523103 16667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:19.523152 16667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:19.523171 16667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:19.523187 16667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:19.523202 16667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:19.528354 16667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:19.529470 16667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:19.530489 16667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:19.539254 16674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:19.539260 16673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:19.539321 16676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:19.539491 16667 server_base.cc:1034] running on GCE node
I20260807 02:31:19.540171 16667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:19.541428 16667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:19.542639 16667 hybrid_clock.cc:648] HybridClock initialized: now 1786069879542608 us; error 40 us; skew 500 ppm
Aug 07 02:31:19 dist-test-slave-g170 krb5kdc[16651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:19.545478 16667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:19.546468 16667 webserver.cc:460] Webserver started at http://127.0.0.1:38385/ using document root <none> and password file <none>
I20260807 02:31:19.547060 16667 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:19.547158 16667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:19.547410 16667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:19.549587 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6"
format_stamp: "Formatted at 2026-08-07 02:31:19 on dist-test-slave-g170"
server_key: "f8b7a81cf9b2bb580cabaa3bbe6f9c37"
server_key_iv: "f3fcbd13bc47a9c85c026b04b21d9224"
server_key_version: "encryptionkey@0"
I20260807 02:31:19.550180 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6"
format_stamp: "Formatted at 2026-08-07 02:31:19 on dist-test-slave-g170"
server_key: "f8b7a81cf9b2bb580cabaa3bbe6f9c37"
server_key_iv: "f3fcbd13bc47a9c85c026b04b21d9224"
server_key_version: "encryptionkey@0"
I20260807 02:31:19.554136 16667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:31:19.556660 16691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:19.558077 16667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:19.558235 16667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6"
format_stamp: "Formatted at 2026-08-07 02:31:19 on dist-test-slave-g170"
server_key: "f8b7a81cf9b2bb580cabaa3bbe6f9c37"
server_key_iv: "f3fcbd13bc47a9c85c026b04b21d9224"
server_key_version: "encryptionkey@0"
I20260807 02:31:19.558352 16667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:19.563117 16667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:19.566118 16667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:19.566319 16667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:19.573503 16667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40691
I20260807 02:31:19.573527 16773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40691 every 8 connection(s)
I20260807 02:31:19.574399 16667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:19.577078 16774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:19.581738 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 16667
I20260807 02:31:19.581897 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:19.582221 17620 external_mini_cluster.cc:1428] Setting key d29d8236d3989172268180119445b61d
I20260807 02:31:19.583084 16774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6: Bootstrap starting.
I20260807 02:31:19.585537 16774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:19.586604 16774 log.cc:824] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:19.587618 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:19.583689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48779 (local address 127.0.0.1:40691)
0807 02:31:19.584120 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:19.584135 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:19.584170 (+ 35us) server_negotiation.cc:407] Connection header received
0807 02:31:19.584815 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:19.584833 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:19.585110 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:19.585446 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:19.586249 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:19.586253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:19.586265 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:19.586276 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:19.586332 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:19.586351 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:19.586469 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:19.586578 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 02:31:19.586744 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":121,"threads_started":1}
I20260807 02:31:19.589018 16774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6: No bootstrap required, opened a new log
I20260807 02:31:19.590663 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:19.593428 16774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260807 02:31:19.593739 16774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:19.593873 16774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f7fa51ef7940a3aca0cbcffe3cf1f6, State: Initialized, Role: FOLLOWER
I20260807 02:31:19.594383 16774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260807 02:31:19.594568 16774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:19.594658 16774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:19.594695 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:19.592752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45823 (local address 127.0.0.1:40691)
0807 02:31:19.592891 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:19.592895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:19.593066 (+ 171us) server_negotiation.cc:407] Connection header received
0807 02:31:19.593197 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:19.593199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:19.593229 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:19.593296 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:19.594197 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:19.594201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:19.594204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:19.594218 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:19.594248 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:19.594263 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:19.594364 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:19.594433 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 02:31:19.594474 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:31:19.594789 16774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:19.595988 16774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260807 02:31:19.596354 16774 leader_election.cc:302] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f7fa51ef7940a3aca0cbcffe3cf1f6; no voters:
I20260807 02:31:19.596684 16774 leader_election.cc:288] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:19.596879 16790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:19.597110 16790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 99f7fa51ef7940a3aca0cbcffe3cf1f6, State: Running, Role: LEADER
I20260807 02:31:19.597415 16790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } }
I20260807 02:31:19.598034 16774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:19.599315 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99f7fa51ef7940a3aca0cbcffe3cf1f6. Latest consensus state: current_term: 1 leader_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } } }
I20260807 02:31:19.599321 16791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f7fa51ef7940a3aca0cbcffe3cf1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40691 } } }
I20260807 02:31:19.599476 16791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:19.599478 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:19.600548 16800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:19.603263 16800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:19.608438 16800 catalog_manager.cc:1347] Generated new cluster ID: 97bbc7d4df9148cea06a6643b44ace25
I20260807 02:31:19.608516 16800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:19.623955 16800 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:19.624339 16800 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:19.631603 16800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99f7fa51ef7940a3aca0cbcffe3cf1f6: Generated new TSK 0
I20260807 02:31:19.632356 16800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:19.642941 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 16667
2026-08-07T02:31:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1654 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:19 dist-test-slave-g170 krb5kdc[16817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:19 dist-test-slave-g170 krb5kdc[16817](info): set up 2 sockets
Aug 07 02:31:19 dist-test-slave-g170 krb5kdc[16817](info): commencing operation
krb5kdc: starting...
W20260807 02:31:20.915181 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:21.053076 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:21.175257 16841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:21.175523 16841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:21.175565 16841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:21.179543 16841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:21.179598 16841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:21.179617 16841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:21.179639 16841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:21.179687 16841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:21.185070 16841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:21.186164 16841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:21.187249 16841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:21.196108 16853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:21.196125 16850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:21.196108 16851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:21.196405 16841 server_base.cc:1034] running on GCE node
I20260807 02:31:21.197027 16841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:21.198093 16841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:21.199265 16841 hybrid_clock.cc:648] HybridClock initialized: now 1786069881199251 us; error 29 us; skew 500 ppm
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:21.206521 16841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:21.207446 16841 webserver.cc:460] Webserver started at http://127.0.0.1:46131/ using document root <none> and password file <none>
I20260807 02:31:21.208065 16841 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:21.208128 16841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:21.208379 16841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:21.210551 16841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "4d3bc55dd7bb493fb770c97c731daf1a"
format_stamp: "Formatted at 2026-08-07 02:31:21 on dist-test-slave-g170"
server_key: "38364bce9d45ec4322f94e36dbe9ed85"
server_key_iv: "766eaa8ca9b0bf8124f03d4f99a3d449"
server_key_version: "encryptionkey@0"
I20260807 02:31:21.211112 16841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "4d3bc55dd7bb493fb770c97c731daf1a"
format_stamp: "Formatted at 2026-08-07 02:31:21 on dist-test-slave-g170"
server_key: "38364bce9d45ec4322f94e36dbe9ed85"
server_key_iv: "766eaa8ca9b0bf8124f03d4f99a3d449"
server_key_version: "encryptionkey@0"
I20260807 02:31:21.214969 16841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:21.217454 16863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:21.218720 16841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:21.218973 16841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "4d3bc55dd7bb493fb770c97c731daf1a"
format_stamp: "Formatted at 2026-08-07 02:31:21 on dist-test-slave-g170"
server_key: "38364bce9d45ec4322f94e36dbe9ed85"
server_key_iv: "766eaa8ca9b0bf8124f03d4f99a3d449"
server_key_version: "encryptionkey@0"
I20260807 02:31:21.219092 16841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:21.233737 16841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:21.237404 16841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:21.237757 16841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:21.246166 16841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33729
I20260807 02:31:21.246196 16945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33729 every 8 connection(s)
I20260807 02:31:21.247231 16841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:21.250276 16947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:21.252823 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 16841
I20260807 02:31:21.253022 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:21.253340 17620 external_mini_cluster.cc:1428] Setting key 121c61e4b76fc66908d3641cf1c3c7af
I20260807 02:31:21.256588 16947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: Bootstrap starting.
I20260807 02:31:21.259012 16947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:21.260000 16947 log.cc:824] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:21.262744 16947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: No bootstrap required, opened a new log
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 02:31:21.266330 16947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } }
I20260807 02:31:21.266570 16947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:21.266642 16947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d3bc55dd7bb493fb770c97c731daf1a, State: Initialized, Role: FOLLOWER
I20260807 02:31:21.267172 16947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } }
I20260807 02:31:21.267336 16947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:21.267428 16947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:21.267556 16947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:21.268513 16947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } }
I20260807 02:31:21.268951 16947 leader_election.cc:302] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d3bc55dd7bb493fb770c97c731daf1a; no voters:
I20260807 02:31:21.269359 16947 leader_election.cc:288] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:21.269505 16954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:21.269773 16954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [term 1 LEADER]: Becoming Leader. State: Replica: 4d3bc55dd7bb493fb770c97c731daf1a, State: Running, Role: LEADER
I20260807 02:31:21.270148 16954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } }
I20260807 02:31:21.270576 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:21.254857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:40327 (local address 127.0.0.1:33729)
0807 02:31:21.255361 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:21.255376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:21.255413 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:31:21.256116 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:21.256135 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:21.256516 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:21.257115 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:21.257976 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:21.259013 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:21.260316 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:21.261066 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:21.261196 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:21.264205 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:21.264210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:21.264224 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:21.264253 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:21.266753 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:21.267303 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:21.267334 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:21.267342 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:21.267532 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:21.267999 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:21.268003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:21.268005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:21.268484 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:21.268904 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:21.269290 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 02:31:21.269587 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":140,"threads_started":1}
I20260807 02:31:21.270742 16947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:21.271396 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d3bc55dd7bb493fb770c97c731daf1a. Latest consensus state: current_term: 1 leader_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } } }
I20260807 02:31:21.271526 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:21.271871 16956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d3bc55dd7bb493fb770c97c731daf1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33729 } } }
I20260807 02:31:21.271966 16956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:21.273916 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 02:31:21.276250 16977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:21.276324 16977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:21.276423 16964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:21.277413 16964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:21.284130 16964 catalog_manager.cc:1347] Generated new cluster ID: b754b7aaaf5b4c0280df7fc0d2507132
I20260807 02:31:21.284204 16964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:21.285087 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:21.276231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:51023 (local address 127.0.0.1:33729)
0807 02:31:21.276353 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:21.276357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:21.276666 (+ 309us) server_negotiation.cc:407] Connection header received
0807 02:31:21.276842 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:21.276845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:21.276895 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:21.277014 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:21.277807 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:21.278395 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:21.279213 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:21.279517 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:21.279614 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:21.281470 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:21.281474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:21.281477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:21.281503 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:21.283157 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:21.283646 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:21.283652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:21.283656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:21.283741 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:21.284088 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:21.284092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:21.284094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:21.284352 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:21.284526 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:21.284836 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 02:31:21.284935 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 02:31:21.310642 16964 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:21.311734 16964 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:21.322165 16964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d3bc55dd7bb493fb770c97c731daf1a: Generated new TSK 0
I20260807 02:31:21.323122 16964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:21.328075 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 16841
W20260807 02:31:21.340077 16814 connection.cc:535] client connection to 127.0.0.1:33729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33729): Connection reset by peer (error 104)
2026-08-07T02:31:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1678 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16993](info): set up 2 sockets
Aug 07 02:31:21 dist-test-slave-g170 krb5kdc[16993](info): commencing operation
krb5kdc: starting...
W20260807 02:31:22.601078 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:22 dist-test-slave-g170 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:22.735388 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:22.855448 17016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:22.855726 17016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:22.855769 17016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:22.859920 17016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:22.859988 17016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:22.860008 17016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:22.860025 17016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:22.860040 17016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:22.865487 17016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:22.866617 17016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:22.867736 17016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:22.876845 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:22.876825 17026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:22.877128 17023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:22.877151 17016 server_base.cc:1034] running on GCE node
I20260807 02:31:22.877830 17016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:22.879086 17016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:22.880295 17016 hybrid_clock.cc:648] HybridClock initialized: now 1786069882880273 us; error 33 us; skew 500 ppm
Aug 07 02:31:22 dist-test-slave-g170 krb5kdc[16993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:22.883581 17016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:22.884599 17016 webserver.cc:460] Webserver started at http://127.0.0.1:43415/ using document root <none> and password file <none>
I20260807 02:31:22.885305 17016 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:22.885406 17016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:22.885675 17016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:22.887877 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "9ff82b84d2d94687a160757d49effbd6"
format_stamp: "Formatted at 2026-08-07 02:31:22 on dist-test-slave-g170"
server_key: "e0ed3874af35e05b70a87cc5a0790cb8"
server_key_iv: "9b122560ba232451a1297c85fdc71938"
server_key_version: "encryptionkey@0"
I20260807 02:31:22.888509 17016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "9ff82b84d2d94687a160757d49effbd6"
format_stamp: "Formatted at 2026-08-07 02:31:22 on dist-test-slave-g170"
server_key: "e0ed3874af35e05b70a87cc5a0790cb8"
server_key_iv: "9b122560ba232451a1297c85fdc71938"
server_key_version: "encryptionkey@0"
I20260807 02:31:22.892601 17016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 02:31:22.895418 17036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:22.896713 17016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 02:31:22.896900 17016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "9ff82b84d2d94687a160757d49effbd6"
format_stamp: "Formatted at 2026-08-07 02:31:22 on dist-test-slave-g170"
server_key: "e0ed3874af35e05b70a87cc5a0790cb8"
server_key_iv: "9b122560ba232451a1297c85fdc71938"
server_key_version: "encryptionkey@0"
I20260807 02:31:22.897043 17016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:22.909058 17016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:22.911979 17016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:22.912204 17016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:22.919936 17016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40215
I20260807 02:31:22.919966 17118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40215 every 8 connection(s)
I20260807 02:31:22.921041 17016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:22.923267 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17016
I20260807 02:31:22.923406 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:22.923692 17620 external_mini_cluster.cc:1428] Setting key cac7125e851fca715a8256ef8a532692
I20260807 02:31:22.924216 17119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:22.930958 17119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6: Bootstrap starting.
I20260807 02:31:22.933061 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:22.925248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:33643 (local address 127.0.0.1:40215)
0807 02:31:22.925674 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:22.925698 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:22.925735 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:31:22.926421 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:22.926438 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:22.926784 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:22.927063 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:22.927488 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:22.928541 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:22.929530 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:22.930034 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:22.930230 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:22.931309 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:22.931314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:22.931326 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:22.931709 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:22.931763 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:22.931787 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:22.931934 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:22.932122 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 02:31:22.932276 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":142,"threads_started":1}
I20260807 02:31:22.933377 17119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:22.934208 17119 log.cc:824] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:22.936098 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:22.936455 17119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6: No bootstrap required, opened a new log
I20260807 02:31:22.940793 17119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260807 02:31:22.941089 17119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:22.941171 17119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff82b84d2d94687a160757d49effbd6, State: Initialized, Role: FOLLOWER
I20260807 02:31:22.941717 17119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260807 02:31:22.941910 17119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:22.942014 17119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:22.942130 17119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:22.942914 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:22.938287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:56905 (local address 127.0.0.1:40215)
0807 02:31:22.938422 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:22.938426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:22.938623 (+ 197us) server_negotiation.cc:407] Connection header received
0807 02:31:22.938795 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:22.938797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:22.938836 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:22.938903 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:22.939227 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:22.939792 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:22.940916 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:22.941267 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:22.941356 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:22.942371 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:22.942378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:22.942388 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:22.942406 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:22.942437 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:22.942451 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:22.942549 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:22.942736 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 02:31:22.942776 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:31:22.943476 17119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260807 02:31:22.943863 17119 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff82b84d2d94687a160757d49effbd6; no voters:
I20260807 02:31:22.944269 17119 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:22.944401 17132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:22.944671 17132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [term 1 LEADER]: Becoming Leader. State: Replica: 9ff82b84d2d94687a160757d49effbd6, State: Running, Role: LEADER
I20260807 02:31:22.945185 17132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260807 02:31:22.945640 17119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:22.946834 17134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff82b84d2d94687a160757d49effbd6. Latest consensus state: current_term: 1 leader_uuid: "9ff82b84d2d94687a160757d49effbd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } } }
I20260807 02:31:22.946973 17134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:22.947336 17133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ff82b84d2d94687a160757d49effbd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff82b84d2d94687a160757d49effbd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } } }
I20260807 02:31:22.947436 17133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:22.947536 17152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:22.950740 17152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:22.956720 17152 catalog_manager.cc:1347] Generated new cluster ID: 3f3eba915ce5458a8894fd20326a5819
I20260807 02:31:22.956827 17152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:22.969287 17152 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:22.970330 17152 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:22.976514 17152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff82b84d2d94687a160757d49effbd6: Generated new TSK 0
I20260807 02:31:22.977504 17152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:22.982894 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17016
W20260807 02:31:23.000352 16989 connection.cc:535] client connection to 127.0.0.1:40215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40215): Connection reset by peer (error 104)
2026-08-07T02:31:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1659 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:23 dist-test-slave-g170 krb5kdc[17177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:23 dist-test-slave-g170 krb5kdc[17177](info): set up 2 sockets
Aug 07 02:31:23 dist-test-slave-g170 krb5kdc[17177](info): commencing operation
krb5kdc: starting...
W20260807 02:31:24.255698 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:24 dist-test-slave-g170 krb5kdc[17177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:24.396329 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:24.522892 17194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:24.523204 17194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:24.523253 17194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:24.527424 17194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:24.527501 17194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:24.527525 17194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:24.527546 17194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:24.527565 17194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:24.533061 17194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:24.534139 17194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:24.535179 17194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:24.544282 17206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:24.544221 17203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:24.544229 17204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:24.544741 17194 server_base.cc:1034] running on GCE node
I20260807 02:31:24.545349 17194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:24.546476 17194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:24.547648 17194 hybrid_clock.cc:648] HybridClock initialized: now 1786069884547629 us; error 38 us; skew 500 ppm
Aug 07 02:31:24 dist-test-slave-g170 krb5kdc[17177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:24.550551 17194 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:24.551455 17194 webserver.cc:460] Webserver started at http://127.0.0.1:41891/ using document root <none> and password file <none>
I20260807 02:31:24.552049 17194 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:24.552112 17194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:24.552337 17194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:24.554517 17194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "27940312f7774e6cb9096c2c7b789353"
format_stamp: "Formatted at 2026-08-07 02:31:24 on dist-test-slave-g170"
server_key: "7ff704594b28eee2b321c3e3c5853e45"
server_key_iv: "08707d9d41fa780ed7227746984a6a84"
server_key_version: "encryptionkey@0"
I20260807 02:31:24.555105 17194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "27940312f7774e6cb9096c2c7b789353"
format_stamp: "Formatted at 2026-08-07 02:31:24 on dist-test-slave-g170"
server_key: "7ff704594b28eee2b321c3e3c5853e45"
server_key_iv: "08707d9d41fa780ed7227746984a6a84"
server_key_version: "encryptionkey@0"
I20260807 02:31:24.558996 17194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 02:31:24.561661 17215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:24.563074 17194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 02:31:24.563258 17194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "27940312f7774e6cb9096c2c7b789353"
format_stamp: "Formatted at 2026-08-07 02:31:24 on dist-test-slave-g170"
server_key: "7ff704594b28eee2b321c3e3c5853e45"
server_key_iv: "08707d9d41fa780ed7227746984a6a84"
server_key_version: "encryptionkey@0"
I20260807 02:31:24.563391 17194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:24.568279 17194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:24.571380 17194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:24.571596 17194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:24.579751 17194 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44307
I20260807 02:31:24.579773 17301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44307 every 8 connection(s)
I20260807 02:31:24.580731 17194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:24.583688 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:24.584576 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17194
I20260807 02:31:24.584751 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:24.585090 17620 external_mini_cluster.cc:1428] Setting key 55dd2e736102c4c8990be9c9efaf146f
I20260807 02:31:24.590555 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353: Bootstrap starting.
I20260807 02:31:24.590852 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:24.586592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:38419 (local address 127.0.0.1:44307)
0807 02:31:24.587079 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:24.587097 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:24.587136 (+ 39us) server_negotiation.cc:407] Connection header received
0807 02:31:24.587832 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:24.587851 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:24.588150 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:24.588489 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:24.589386 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:24.589391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:24.589403 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:24.589414 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:24.589468 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:24.589488 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:24.589606 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:24.589813 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 02:31:24.590013 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":170,"threads_started":1}
I20260807 02:31:24.592947 17302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:24.593677 17302 log.cc:824] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:24.593904 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:24.596061 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353: No bootstrap required, opened a new log
I20260807 02:31:24.598095 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:24.596170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:45195 (local address 127.0.0.1:44307)
0807 02:31:24.596281 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:24.596285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:24.596566 (+ 281us) server_negotiation.cc:407] Connection header received
0807 02:31:24.596687 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:24.596689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:24.596743 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:24.596860 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:24.597554 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:24.597557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:24.597559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:24.597569 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:24.597595 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:24.597605 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:24.597676 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:24.597912 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 02:31:24.597963 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 02:31:24.600260 17302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } }
I20260807 02:31:24.600518 17302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:24.600580 17302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27940312f7774e6cb9096c2c7b789353, State: Initialized, Role: FOLLOWER
I20260807 02:31:24.601091 17302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } }
I20260807 02:31:24.601249 17302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:24.601317 17302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:24.601423 17302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:24.602321 17302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } }
I20260807 02:31:24.602679 17302 leader_election.cc:302] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27940312f7774e6cb9096c2c7b789353; no voters:
I20260807 02:31:24.602994 17302 leader_election.cc:288] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:24.603127 17317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:24.603559 17317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [term 1 LEADER]: Becoming Leader. State: Replica: 27940312f7774e6cb9096c2c7b789353, State: Running, Role: LEADER
I20260807 02:31:24.603947 17317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } }
I20260807 02:31:24.604600 17302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:24.606015 17318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27940312f7774e6cb9096c2c7b789353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } } }
I20260807 02:31:24.606048 17319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27940312f7774e6cb9096c2c7b789353. Latest consensus state: current_term: 1 leader_uuid: "27940312f7774e6cb9096c2c7b789353" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27940312f7774e6cb9096c2c7b789353" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44307 } } }
I20260807 02:31:24.606221 17319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:24.606215 17318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:24.606632 17334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:24.610203 17334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:24.616061 17334 catalog_manager.cc:1347] Generated new cluster ID: 128c44ebb6e5484e89b0c737f0ce9a9d
I20260807 02:31:24.616189 17334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:24.626955 17334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:24.627280 17334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:24.635118 17334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27940312f7774e6cb9096c2c7b789353: Generated new TSK 0
I20260807 02:31:24.635921 17334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:24.648504 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17194
2026-08-07T02:31:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1665 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:24 dist-test-slave-g170 krb5kdc[17349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:24 dist-test-slave-g170 krb5kdc[17349](info): set up 2 sockets
Aug 07 02:31:24 dist-test-slave-g170 krb5kdc[17349](info): commencing operation
krb5kdc: starting...
W20260807 02:31:25.942361 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:26.090293 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:26.207643 17371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:26.208048 17371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:26.208099 17371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:26.212755 17371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:26.213049 17371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:26.213124 17371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:26.213186 17371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:26.213224 17371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:26.219017 17371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44267
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:26.220114 17371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:26.221182 17371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:26.230127 17380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:26.230239 17371 server_base.cc:1034] running on GCE node
W20260807 02:31:26.230062 17383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:26.230060 17381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:26.230973 17371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:26.232131 17371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:26.233296 17371 hybrid_clock.cc:648] HybridClock initialized: now 1786069886233269 us; error 44 us; skew 500 ppm
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:26.241986 17371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:26.242988 17371 webserver.cc:460] Webserver started at http://127.0.0.1:38067/ using document root <none> and password file <none>
I20260807 02:31:26.243588 17371 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:26.243685 17371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:26.243942 17371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:26.247102 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "21e5a37810224d8da25d25afc7f8d1d0"
format_stamp: "Formatted at 2026-08-07 02:31:26 on dist-test-slave-g170"
server_key: "57c3f37afe4d9aa17c1a1041cae73711"
server_key_iv: "bc4f3c99c06b204570de657c5b019ce1"
server_key_version: "encryptionkey@0"
I20260807 02:31:26.247828 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "21e5a37810224d8da25d25afc7f8d1d0"
format_stamp: "Formatted at 2026-08-07 02:31:26 on dist-test-slave-g170"
server_key: "57c3f37afe4d9aa17c1a1041cae73711"
server_key_iv: "bc4f3c99c06b204570de657c5b019ce1"
server_key_version: "encryptionkey@0"
I20260807 02:31:26.252985 17371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 02:31:26.255771 17393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:26.257205 17371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:26.257351 17371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "21e5a37810224d8da25d25afc7f8d1d0"
format_stamp: "Formatted at 2026-08-07 02:31:26 on dist-test-slave-g170"
server_key: "57c3f37afe4d9aa17c1a1041cae73711"
server_key_iv: "bc4f3c99c06b204570de657c5b019ce1"
server_key_version: "encryptionkey@0"
I20260807 02:31:26.257464 17371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:26.283798 17371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:26.287088 17371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:26.287319 17371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:26.295068 17371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44267
I20260807 02:31:26.295089 17473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44267 every 8 connection(s)
I20260807 02:31:26.296063 17371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:26.298069 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17371
I20260807 02:31:26.298218 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:26.298602 17620 external_mini_cluster.cc:1428] Setting key 7de9d950d467b08b56303a6be0cd1d3b
I20260807 02:31:26.299381 17474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:26.305873 17474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: Bootstrap starting.
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 02:31:26.308444 17474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:26.309233 17474 log.cc:824] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:26.311210 17474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: No bootstrap required, opened a new log
I20260807 02:31:26.313490 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:26.299966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:58907 (local address 127.0.0.1:44267)
0807 02:31:26.300382 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:26.300397 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:26.300432 (+ 35us) server_negotiation.cc:407] Connection header received
0807 02:31:26.301190 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:26.301211 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:26.301555 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:26.301906 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:26.302731 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:26.303703 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:26.304582 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:26.305121 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:26.305300 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:26.308227 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:26.308236 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:26.308257 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:26.308299 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:26.310345 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:26.310890 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:26.310895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:26.310899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:26.311009 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:26.311371 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:26.311375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:26.311377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:26.311761 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:26.311988 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:26.312335 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 02:31:26.312564 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
I20260807 02:31:26.315120 17474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260807 02:31:26.315387 17474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:26.315454 17474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e5a37810224d8da25d25afc7f8d1d0, State: Initialized, Role: FOLLOWER
I20260807 02:31:26.315941 17474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260807 02:31:26.316107 17474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:26.316174 17474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:26.316282 17474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:26.316550 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:26.317459 17474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260807 02:31:26.317828 17474 leader_election.cc:302] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21e5a37810224d8da25d25afc7f8d1d0; no voters:
I20260807 02:31:26.318235 17474 leader_election.cc:288] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:26.318326 17490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:26.318650 17490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [term 1 LEADER]: Becoming Leader. State: Replica: 21e5a37810224d8da25d25afc7f8d1d0, State: Running, Role: LEADER
I20260807 02:31:26.319038 17490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260807 02:31:26.319545 17474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:26.320912 17495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21e5a37810224d8da25d25afc7f8d1d0. Latest consensus state: current_term: 1 leader_uuid: "21e5a37810224d8da25d25afc7f8d1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } } }
I20260807 02:31:26.320925 17493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21e5a37810224d8da25d25afc7f8d1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e5a37810224d8da25d25afc7f8d1d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } } }
I20260807 02:31:26.321135 17493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:26.321170 17495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0 [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:26.324721 17515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:26.324843 17515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:26.324944 17508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:26.325811 17508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:26.328905 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:26.319046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:58961 (local address 127.0.0.1:44267)
0807 02:31:26.319147 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:26.319151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:26.320388 (+ 1237us) server_negotiation.cc:407] Connection header received
0807 02:31:26.320594 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:26.320599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:26.320653 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:26.320708 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:26.321758 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:26.322482 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:26.323584 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:26.323841 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:26.323953 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:26.325501 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:26.325505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:26.325508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:26.325535 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:26.326984 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:26.327478 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:26.327481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:26.327483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:26.327592 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:26.328001 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:26.328005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:26.328007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:26.328179 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:26.328344 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:26.328614 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 02:31:26.328707 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 02:31:26.331461 17508 catalog_manager.cc:1347] Generated new cluster ID: 7f3d76b1ee704a1290476b459f41bcc1
I20260807 02:31:26.331525 17508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:26.352447 17508 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:26.353477 17508 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:26.365332 17508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21e5a37810224d8da25d25afc7f8d1d0: Generated new TSK 0
I20260807 02:31:26.366132 17508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:26.376941 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17371
W20260807 02:31:26.391377 17347 connection.cc:535] client connection to 127.0.0.1:44267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44267): Connection reset by peer (error 104)
2026-08-07T02:31:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1724 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17524](info): set up 2 sockets
Aug 07 02:31:26 dist-test-slave-g170 krb5kdc[17524](info): commencing operation
krb5kdc: starting...
W20260807 02:31:27.682938 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:27 dist-test-slave-g170 krb5kdc[17524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:27.820111 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:27.939852 17540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:27.940178 17540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:27.940256 17540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:27.944269 17540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:27.944370 17540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:27.944410 17540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:27.944470 17540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:27.944527 17540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:27.949926 17540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:27.951153 17540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:27.952271 17540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:27.961248 17547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:27.961264 17549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:27.961301 17546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:27.961524 17540 server_base.cc:1034] running on GCE node
I20260807 02:31:27.962231 17540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:27.963523 17540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:27.964854 17540 hybrid_clock.cc:648] HybridClock initialized: now 1786069887964830 us; error 32 us; skew 500 ppm
Aug 07 02:31:27 dist-test-slave-g170 krb5kdc[17524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:27.968057 17540 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:27.969264 17540 webserver.cc:460] Webserver started at http://127.0.0.1:37183/ using document root <none> and password file <none>
I20260807 02:31:27.969937 17540 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:27.970041 17540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:27.970302 17540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:27.972532 17540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac"
format_stamp: "Formatted at 2026-08-07 02:31:27 on dist-test-slave-g170"
server_key: "c0a43e29f482e652c3b982ce4af9e5a7"
server_key_iv: "957406ee0ba69ada167f852f3a2760de"
server_key_version: "encryptionkey@0"
I20260807 02:31:27.973196 17540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac"
format_stamp: "Formatted at 2026-08-07 02:31:27 on dist-test-slave-g170"
server_key: "c0a43e29f482e652c3b982ce4af9e5a7"
server_key_iv: "957406ee0ba69ada167f852f3a2760de"
server_key_version: "encryptionkey@0"
I20260807 02:31:27.977314 17540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:31:27.980158 17556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:27.981779 17540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:27.981974 17540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac"
format_stamp: "Formatted at 2026-08-07 02:31:27 on dist-test-slave-g170"
server_key: "c0a43e29f482e652c3b982ce4af9e5a7"
server_key_iv: "957406ee0ba69ada167f852f3a2760de"
server_key_version: "encryptionkey@0"
I20260807 02:31:27.982118 17540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:27.995231 17540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:27.998643 17540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:27.998893 17540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:28.006752 17540 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35377
I20260807 02:31:28.006800 17645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35377 every 8 connection(s)
I20260807 02:31:28.007808 17540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:28.008803 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17540
I20260807 02:31:28.008950 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:28.009296 17620 external_mini_cluster.cc:1428] Setting key ea8e1403dea8cc78e993a8e460d3cf8d
I20260807 02:31:28.012298 17646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:28.018509 17646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac: Bootstrap starting.
I20260807 02:31:28.018610 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:28.010705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:39401 (local address 127.0.0.1:35377)
0807 02:31:28.011149 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:28.011169 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:28.011207 (+ 38us) server_negotiation.cc:407] Connection header received
0807 02:31:28.011897 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:28.011915 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:28.012298 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:28.012701 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:28.013156 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:28.014165 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:28.015084 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:28.015572 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:28.015749 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:28.016743 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:28.016748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:28.016787 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:28.017273 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:28.017351 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:28.017388 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:28.017532 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:28.017717 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 02:31:28.017867 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":115,"threads_started":1}
I20260807 02:31:28.021211 17646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:28.021781 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:28.022140 17646 log.cc:824] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:28.024473 17646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac: No bootstrap required, opened a new log
I20260807 02:31:28.028191 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:28.023839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:56285 (local address 127.0.0.1:35377)
0807 02:31:28.023973 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:28.023977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:28.024180 (+ 203us) server_negotiation.cc:407] Connection header received
0807 02:31:28.024313 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:28.024315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:28.024353 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:28.024418 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:28.024699 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:28.025255 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:28.026446 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:28.026678 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:28.026789 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:28.027680 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:28.027684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:28.027686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:28.027698 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:28.027724 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:28.027735 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:28.027816 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:28.027985 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 02:31:28.028042 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:31:28.029222 17646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260807 02:31:28.029481 17646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:28.029567 17646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9a1e1c2b8149d0adb0cb9d156986ac, State: Initialized, Role: FOLLOWER
I20260807 02:31:28.030006 17646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260807 02:31:28.030158 17646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:28.030248 17646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:28.030357 17646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:28.031325 17646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260807 02:31:28.031679 17646 leader_election.cc:302] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d9a1e1c2b8149d0adb0cb9d156986ac; no voters:
I20260807 02:31:28.032017 17646 leader_election.cc:288] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:28.032143 17662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:28.032426 17662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [term 1 LEADER]: Becoming Leader. State: Replica: 2d9a1e1c2b8149d0adb0cb9d156986ac, State: Running, Role: LEADER
I20260807 02:31:28.032825 17662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } }
I20260807 02:31:28.033418 17646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:28.034554 17664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } } }
I20260807 02:31:28.034610 17668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d9a1e1c2b8149d0adb0cb9d156986ac. Latest consensus state: current_term: 1 leader_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9a1e1c2b8149d0adb0cb9d156986ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35377 } } }
I20260807 02:31:28.034763 17664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:28.034770 17668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:28.035202 17678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:28.038518 17678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:28.044140 17678 catalog_manager.cc:1347] Generated new cluster ID: 9374eac36eae41edb933770b3ef0dcfa
I20260807 02:31:28.044209 17678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:28.061873 17678 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:28.062844 17678 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:28.069599 17678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d9a1e1c2b8149d0adb0cb9d156986ac: Generated new TSK 0
I20260807 02:31:28.070344 17678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:28.078323 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17540
W20260807 02:31:28.092278 17522 connection.cc:535] client connection to 127.0.0.1:35377 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35377): Connection reset by peer (error 104)
2026-08-07T02:31:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1701 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:28 dist-test-slave-g170 krb5kdc[17692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:28 dist-test-slave-g170 krb5kdc[17692](info): set up 2 sockets
Aug 07 02:31:28 dist-test-slave-g170 krb5kdc[17692](info): commencing operation
krb5kdc: starting...
W20260807 02:31:29.361874 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:29 dist-test-slave-g170 krb5kdc[17692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 02:31:29.505266 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:29.627774 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:29.628078 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:29.628123 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:29.632335 17710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:29.632457 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:29.632495 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:29.632537 17710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:29.632568 17710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:29.638159 17710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:29.639365 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:29.640452 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:29.649574 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:29.649602 17718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:29.649623 17720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:29.650079 17710 server_base.cc:1034] running on GCE node
I20260807 02:31:29.650596 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:29.651665 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:29.652925 17710 hybrid_clock.cc:648] HybridClock initialized: now 1786069889652907 us; error 28 us; skew 500 ppm
Aug 07 02:31:29 dist-test-slave-g170 krb5kdc[17692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:29.656049 17710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 02:31:29.657083 17710 webserver.cc:460] Webserver started at http://127.0.0.1:33321/ using document root <none> and password file <none>
I20260807 02:31:29.657686 17710 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:29.657749 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:29.658001 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:29.660231 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "57bc0bee79934199b5fd12e86b7a06df"
format_stamp: "Formatted at 2026-08-07 02:31:29 on dist-test-slave-g170"
server_key: "c09f9125e59050e0a57effbd3e1e7780"
server_key_iv: "8068665690dbc86171ad6781b4602101"
server_key_version: "encryptionkey@0"
I20260807 02:31:29.660923 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "57bc0bee79934199b5fd12e86b7a06df"
format_stamp: "Formatted at 2026-08-07 02:31:29 on dist-test-slave-g170"
server_key: "c09f9125e59050e0a57effbd3e1e7780"
server_key_iv: "8068665690dbc86171ad6781b4602101"
server_key_version: "encryptionkey@0"
I20260807 02:31:29.665077 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 02:31:29.667797 17736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:29.669492 17710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:29.669672 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "57bc0bee79934199b5fd12e86b7a06df"
format_stamp: "Formatted at 2026-08-07 02:31:29 on dist-test-slave-g170"
server_key: "c09f9125e59050e0a57effbd3e1e7780"
server_key_iv: "8068665690dbc86171ad6781b4602101"
server_key_version: "encryptionkey@0"
I20260807 02:31:29.669806 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:29.674803 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:29.678273 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:29.678496 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:29.686342 17710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45719
I20260807 02:31:29.686360 17812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45719 every 8 connection(s)
I20260807 02:31:29.687315 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:29.690516 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:29.693624 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17710
I20260807 02:31:29.693773 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:29.694051 17620 external_mini_cluster.cc:1428] Setting key eab5bb0fcfba7aca8f54d59714345daa
I20260807 02:31:29.696918 17813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df: Bootstrap starting.
I20260807 02:31:29.699846 17813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:29.699905 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:29.695579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:53489 (local address 127.0.0.1:45719)
0807 02:31:29.696026 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:29.696052 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:29.696098 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:31:29.696980 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:29.697008 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:29.697371 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:29.697735 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:29.698460 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:29.698466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:29.698482 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:29.698495 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:29.698557 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:29.698586 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:29.698729 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:29.698805 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 02:31:29.698966 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":118,"threads_started":1}
I20260807 02:31:29.700680 17813 log.cc:824] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:29.702935 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:29.703126 17813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df: No bootstrap required, opened a new log
I20260807 02:31:29.707085 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:29.705154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.221:53701 (local address 127.0.0.1:45719)
0807 02:31:29.705290 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:29.705295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:29.705497 (+ 202us) server_negotiation.cc:407] Connection header received
0807 02:31:29.705622 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:29.705624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:29.705655 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:29.705751 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:29.706532 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:29.706535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:29.706537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:29.706548 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:29.706572 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:29.706582 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:29.706652 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:29.706840 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 02:31:29.706924 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:31:29.707809 17813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } }
I20260807 02:31:29.708117 17813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:29.708179 17813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bc0bee79934199b5fd12e86b7a06df, State: Initialized, Role: FOLLOWER
I20260807 02:31:29.708624 17813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } }
I20260807 02:31:29.708815 17813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:29.708887 17813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:29.708993 17813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:29.710496 17813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } }
I20260807 02:31:29.710886 17813 leader_election.cc:302] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57bc0bee79934199b5fd12e86b7a06df; no voters:
I20260807 02:31:29.711243 17813 leader_election.cc:288] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:29.711426 17841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:29.711692 17841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [term 1 LEADER]: Becoming Leader. State: Replica: 57bc0bee79934199b5fd12e86b7a06df, State: Running, Role: LEADER
I20260807 02:31:29.712064 17841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } }
I20260807 02:31:29.712613 17813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:29.713949 17842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57bc0bee79934199b5fd12e86b7a06df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } } }
I20260807 02:31:29.714087 17842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:29.714453 17843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57bc0bee79934199b5fd12e86b7a06df. Latest consensus state: current_term: 1 leader_uuid: "57bc0bee79934199b5fd12e86b7a06df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bc0bee79934199b5fd12e86b7a06df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45719 } } }
I20260807 02:31:29.714524 17854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:29.714555 17843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:29.718060 17854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:29.723922 17854 catalog_manager.cc:1347] Generated new cluster ID: dfae4f201c754c65929569987e43f270
I20260807 02:31:29.724004 17854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:29.742274 17854 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:29.742708 17854 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:29.754196 17854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57bc0bee79934199b5fd12e86b7a06df: Generated new TSK 0
I20260807 02:31:29.755007 17854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:29.762714 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17710
2026-08-07T02:31:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:29 dist-test-slave-g170 krb5kdc[17873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:29 dist-test-slave-g170 krb5kdc[17873](info): set up 2 sockets
Aug 07 02:31:29 dist-test-slave-g170 krb5kdc[17873](info): commencing operation
krb5kdc: starting...
W20260807 02:31:31.017164 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[17873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:31.168931 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37059
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:37059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:31.283402 17892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:31.283674 17892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:31.283716 17892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:31.287748 17892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:31.287806 17892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:31.287827 17892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:31.287842 17892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:31.287886 17892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:31.293404 17892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:37059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:37059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:31.294649 17892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:31.295851 17892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:31.305805 17902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:31.305843 17901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:31.305811 17905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:31.306175 17892 server_base.cc:1034] running on GCE node
I20260807 02:31:31.306838 17892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:31.307876 17892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:31.309059 17892 hybrid_clock.cc:648] HybridClock initialized: now 1786069891309041 us; error 28 us; skew 500 ppm
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[17873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069891, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:31.317759 17892 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:31.318789 17892 webserver.cc:460] Webserver started at http://127.17.53.62:33157/ using document root <none> and password file <none>
I20260807 02:31:31.319387 17892 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:31.319451 17892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:31.319697 17892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:31.321926 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "d25b5b727983491589e97f966a695d9b"
format_stamp: "Formatted at 2026-08-07 02:31:31 on dist-test-slave-g170"
server_key: "ac82f76a45db57a7503d396ede1343ea"
server_key_iv: "a53b01a4b87cd35ee43b940e7fed91cb"
server_key_version: "encryptionkey@0"
I20260807 02:31:31.322515 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "d25b5b727983491589e97f966a695d9b"
format_stamp: "Formatted at 2026-08-07 02:31:31 on dist-test-slave-g170"
server_key: "ac82f76a45db57a7503d396ede1343ea"
server_key_iv: "a53b01a4b87cd35ee43b940e7fed91cb"
server_key_version: "encryptionkey@0"
I20260807 02:31:31.326514 17892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:31.329315 17915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:31.330614 17892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:31.330798 17892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "d25b5b727983491589e97f966a695d9b"
format_stamp: "Formatted at 2026-08-07 02:31:31 on dist-test-slave-g170"
server_key: "ac82f76a45db57a7503d396ede1343ea"
server_key_iv: "a53b01a4b87cd35ee43b940e7fed91cb"
server_key_version: "encryptionkey@0"
I20260807 02:31:31.330920 17892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:31.341392 17892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:31.344732 17892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:31.345005 17892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:31.352861 17892 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:37059
I20260807 02:31:31.352878 17994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:37059 every 8 connection(s)
I20260807 02:31:31.353883 17892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:31.356499 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 17892
I20260807 02:31:31.356662 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:31.356938 17995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:31.357050 17620 external_mini_cluster.cc:1428] Setting key 86a8dd406ff17d8d7a171344f43969c0
I20260807 02:31:31.364243 17995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b: Bootstrap starting.
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[17873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:31.367200 17995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:31.367967 17995 log.cc:824] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:31.370579 17995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b: No bootstrap required, opened a new log
I20260807 02:31:31.373262 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:31.358500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60531 (local address 127.17.53.62:37059)
0807 02:31:31.358956 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:31.358972 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:31.359009 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:31:31.359620 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:31.359639 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:31.359982 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:31.360273 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:31.361306 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:31.362507 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:31.363770 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:31.364245 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:31.364421 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:31.367356 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:31.367362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:31.367376 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:31.367411 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:31.369572 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:31.370258 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:31.370266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:31.370273 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:31.370417 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:31.370863 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:31.370866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:31.370867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:31.371376 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:31.371588 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:31.371966 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 02:31:31.372192 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":164,"threads_started":1}
I20260807 02:31:31.375164 17995 raft_consensus.cc:348] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } }
I20260807 02:31:31.375454 17995 raft_consensus.cc:374] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:31.375546 17995 raft_consensus.cc:729] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25b5b727983491589e97f966a695d9b, State: Initialized, Role: FOLLOWER
I20260807 02:31:31.375988 17995 consensus_queue.cc:260] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } }
I20260807 02:31:31.376150 17995 raft_consensus.cc:388] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:31.376217 17995 raft_consensus.cc:482] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:31.376330 17995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:31.376444 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:31.377375 17995 raft_consensus.cc:504] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } }
I20260807 02:31:31.377766 17995 leader_election.cc:302] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25b5b727983491589e97f966a695d9b; no voters:
I20260807 02:31:31.378134 17995 leader_election.cc:288] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:31.378206 18006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:31.378443 18006 raft_consensus.cc:686] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [term 1 LEADER]: Becoming Leader. State: Replica: d25b5b727983491589e97f966a695d9b, State: Running, Role: LEADER
I20260807 02:31:31.378803 18006 consensus_queue.cc:237] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } }
I20260807 02:31:31.379499 17995 sys_catalog.cc:564] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:31.380539 18009 sys_catalog.cc:455] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d25b5b727983491589e97f966a695d9b. Latest consensus state: current_term: 1 leader_uuid: "d25b5b727983491589e97f966a695d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } } }
I20260807 02:31:31.380677 18009 sys_catalog.cc:458] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:31.381026 18007 sys_catalog.cc:455] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d25b5b727983491589e97f966a695d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25b5b727983491589e97f966a695d9b" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 37059 } } }
I20260807 02:31:31.381121 18007 sys_catalog.cc:458] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:31.382215 18019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:31.385175 18019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:31.388118 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:31.378942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38265 (local address 127.17.53.62:37059)
0807 02:31:31.379062 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:31.379066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:31.379242 (+ 176us) server_negotiation.cc:407] Connection header received
0807 02:31:31.379390 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:31.379393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:31.379439 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:31.379531 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:31.380341 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:31.381121 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:31.382270 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:31.382638 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:31.382760 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:31.384261 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:31.384267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:31.384270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:31.384296 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:31.386121 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:31.386726 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:31.386730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:31.386732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:31.386826 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:31.387225 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:31.387233 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:31.387235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:31.387413 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:31.387573 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:31.387864 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 02:31:31.387964 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 02:31:31.391507 18019 catalog_manager.cc:1347] Generated new cluster ID: d6e9ccc8299544d5b41832a646f14ca0
I20260807 02:31:31.391629 18019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:31.404454 18019 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:31.405459 18019 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:31.412505 18019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d25b5b727983491589e97f966a695d9b: Generated new TSK 0
I20260807 02:31:31.413357 18019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:31.423498 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 17892
W20260807 02:31:31.441179 17869 connection.cc:535] client connection to 127.17.53.62:37059 recv error: Network error: failed to read from TLS socket (remote: 127.17.53.62:37059): Connection reset by peer (error 104)
2026-08-07T02:31:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[18034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[18034](info): set up 2 sockets
Aug 07 02:31:31 dist-test-slave-g170 krb5kdc[18034](info): commencing operation
krb5kdc: starting...
W20260807 02:31:32.759749 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:32 dist-test-slave-g170 krb5kdc[18034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:32.891566 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46565
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:46565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:33.005245 18052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:33.005564 18052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:33.005640 18052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:33.009627 18052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:33.009733 18052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:33.009774 18052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:33.009832 18052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:33.009873 18052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:33.015206 18052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:46565
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:33.016330 18052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:33.017478 18052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:33.026299 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:33.026345 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:33.026304 18060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:33.026715 18052 server_base.cc:1034] running on GCE node
I20260807 02:31:33.027374 18052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:33.028545 18052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:33.029739 18052 hybrid_clock.cc:648] HybridClock initialized: now 1786069893029721 us; error 35 us; skew 500 ppm
Aug 07 02:31:33 dist-test-slave-g170 krb5kdc[18034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069893, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:33.032723 18052 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:33.033763 18052 webserver.cc:460] Webserver started at http://127.17.53.62:43061/ using document root <none> and password file <none>
I20260807 02:31:33.034374 18052 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:33.034471 18052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:33.034739 18052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:33.036934 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "8b1193e2692948bf917f21371bcf4675"
format_stamp: "Formatted at 2026-08-07 02:31:33 on dist-test-slave-g170"
server_key: "dd358cd6af556bc2272c6a9e7dd31c80"
server_key_iv: "5dc6680283e42738f0ec21c549976968"
server_key_version: "encryptionkey@0"
I20260807 02:31:33.037521 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "8b1193e2692948bf917f21371bcf4675"
format_stamp: "Formatted at 2026-08-07 02:31:33 on dist-test-slave-g170"
server_key: "dd358cd6af556bc2272c6a9e7dd31c80"
server_key_iv: "5dc6680283e42738f0ec21c549976968"
server_key_version: "encryptionkey@0"
I20260807 02:31:33.041414 18052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 02:31:33.043917 18073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:33.045357 18052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:33.045509 18052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "8b1193e2692948bf917f21371bcf4675"
format_stamp: "Formatted at 2026-08-07 02:31:33 on dist-test-slave-g170"
server_key: "dd358cd6af556bc2272c6a9e7dd31c80"
server_key_iv: "5dc6680283e42738f0ec21c549976968"
server_key_version: "encryptionkey@0"
I20260807 02:31:33.045620 18052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:33.061861 18052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:33.064990 18052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:33.065200 18052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:33.072732 18052 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:46565
I20260807 02:31:33.072818 18158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:46565 every 8 connection(s)
I20260807 02:31:33.073791 18052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:33.076682 18159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:33.078536 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18052
I20260807 02:31:33.078707 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:33.078980 17620 external_mini_cluster.cc:1428] Setting key f71fa6fc857f41e80d0640b457f936aa
I20260807 02:31:33.083310 18159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: Bootstrap starting.
I20260807 02:31:33.085646 18159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:33.086400 18159 log.cc:824] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:33.088306 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:33.080414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60769 (local address 127.17.53.62:46565)
0807 02:31:33.081019 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:33.081048 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:33.081094 (+ 46us) server_negotiation.cc:407] Connection header received
0807 02:31:33.081738 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:33.081762 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:33.082091 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:33.082434 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:33.082858 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:33.083880 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:33.084801 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:33.085261 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:33.085446 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:33.086365 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:33.086370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:33.086384 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:33.086872 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:33.086939 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:33.086973 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:33.087124 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:33.087367 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 02:31:33.087541 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":162,"threads_started":1}
I20260807 02:31:33.088516 18159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: No bootstrap required, opened a new log
I20260807 02:31:33.091440 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:33.092842 18159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } }
I20260807 02:31:33.093128 18159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:33.093194 18159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1193e2692948bf917f21371bcf4675, State: Initialized, Role: FOLLOWER
I20260807 02:31:33.093628 18159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } }
I20260807 02:31:33.093822 18159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:33.093896 18159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:33.094039 18159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:33.094901 18159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } }
I20260807 02:31:33.095237 18159 leader_election.cc:302] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b1193e2692948bf917f21371bcf4675; no voters:
I20260807 02:31:33.095572 18159 leader_election.cc:288] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:33.095646 18176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:33.095857 18176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1193e2692948bf917f21371bcf4675, State: Running, Role: LEADER
I20260807 02:31:33.096202 18176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } }
I20260807 02:31:33.096968 18159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:33.099462 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:33.093781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43219 (local address 127.17.53.62:46565)
0807 02:31:33.093897 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:33.093903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:33.094133 (+ 230us) server_negotiation.cc:407] Connection header received
0807 02:31:33.094276 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:33.094279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:33.094319 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:33.094409 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:33.094669 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:33.095285 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:33.096366 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:33.096733 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:33.096853 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:33.098684 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:33.098690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:33.098692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:33.098706 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:33.098734 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:33.098748 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:33.098836 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:33.099247 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 02:31:33.099291 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:31:33.100878 18179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b1193e2692948bf917f21371bcf4675. Latest consensus state: current_term: 1 leader_uuid: "8b1193e2692948bf917f21371bcf4675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } } }
I20260807 02:31:33.101074 18179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:33.101434 18177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b1193e2692948bf917f21371bcf4675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b1193e2692948bf917f21371bcf4675" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46565 } } }
I20260807 02:31:33.101557 18177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675 [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:33.103636 18197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:33.103701 18197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:33.103793 18199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:33.104499 18199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:33.109644 18199 catalog_manager.cc:1347] Generated new cluster ID: 36d3c5a9947b425ba5bfa257307b2fd4
I20260807 02:31:33.109709 18199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:33.128341 18199 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:33.129449 18199 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:33.134078 18199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b1193e2692948bf917f21371bcf4675: Generated new TSK 0
I20260807 02:31:33.134869 18199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:33.147825 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18052
W20260807 02:31:33.162101 18030 connection.cc:535] client connection to 127.17.53.62:46565 recv error: Network error: failed to read from TLS socket (remote: 127.17.53.62:46565): Connection reset by peer (error 104)
2026-08-07T02:31:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1720 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:33 dist-test-slave-g170 krb5kdc[18204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:33 dist-test-slave-g170 krb5kdc[18204](info): set up 2 sockets
Aug 07 02:31:33 dist-test-slave-g170 krb5kdc[18204](info): commencing operation
krb5kdc: starting...
W20260807 02:31:34.416275 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:34 dist-test-slave-g170 krb5kdc[18204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:34.544529 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35503
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:34.663009 18227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:34.663367 18227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:34.663479 18227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:34.667646 18227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:34.667752 18227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:34.667791 18227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:34.667847 18227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:34.667886 18227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:34.674055 18227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:34.675274 18227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:34.676402 18227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:34.685283 18232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:34.685267 18235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:34.685272 18233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:34.685742 18227 server_base.cc:1034] running on GCE node
I20260807 02:31:34.686261 18227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:34.687476 18227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:34.688647 18227 hybrid_clock.cc:648] HybridClock initialized: now 1786069894688631 us; error 25 us; skew 500 ppm
Aug 07 02:31:34 dist-test-slave-g170 krb5kdc[18204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069894, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:34.691458 18227 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:34.692399 18227 webserver.cc:460] Webserver started at http://127.17.53.62:38135/ using document root <none> and password file <none>
I20260807 02:31:34.693048 18227 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:34.693114 18227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:34.693354 18227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:34.695464 18227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4"
format_stamp: "Formatted at 2026-08-07 02:31:34 on dist-test-slave-g170"
server_key: "6932f3d8fa878f1d68a1567ac8f80e4b"
server_key_iv: "5cd2403584df48daa2599fca3fc673c8"
server_key_version: "encryptionkey@0"
I20260807 02:31:34.696046 18227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4"
format_stamp: "Formatted at 2026-08-07 02:31:34 on dist-test-slave-g170"
server_key: "6932f3d8fa878f1d68a1567ac8f80e4b"
server_key_iv: "5cd2403584df48daa2599fca3fc673c8"
server_key_version: "encryptionkey@0"
I20260807 02:31:34.699867 18227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:34.702488 18247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:34.703814 18227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:31:34.703970 18227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4"
format_stamp: "Formatted at 2026-08-07 02:31:34 on dist-test-slave-g170"
server_key: "6932f3d8fa878f1d68a1567ac8f80e4b"
server_key_iv: "5cd2403584df48daa2599fca3fc673c8"
server_key_version: "encryptionkey@0"
I20260807 02:31:34.704079 18227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:34.708904 18227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:34.712386 18227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:34.712617 18227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:34.721659 18227 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35503
I20260807 02:31:34.721678 18337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35503 every 8 connection(s)
I20260807 02:31:34.722713 18227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:34.726022 18338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:34.727233 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18227
I20260807 02:31:34.727399 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:34.727705 17620 external_mini_cluster.cc:1428] Setting key 4318d9f2d0ada537428b7c50e2d22461
I20260807 02:31:34.733297 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: Bootstrap starting.
I20260807 02:31:34.733721 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:34.729251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36419 (local address 127.17.53.62:35503)
0807 02:31:34.729699 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:34.729720 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:34.729765 (+ 45us) server_negotiation.cc:407] Connection header received
0807 02:31:34.730604 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:34.730627 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:34.731000 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:34.731384 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:34.732164 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:34.732172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:34.732191 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:34.732205 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:34.732274 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:34.732312 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:34.732458 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:34.732529 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 02:31:34.732679 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260807 02:31:34.735668 18338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:34.736604 18338 log.cc:824] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:34.738057 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:34.738734 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: No bootstrap required, opened a new log
I20260807 02:31:34.741988 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:34.740285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50659 (local address 127.17.53.62:35503)
0807 02:31:34.740425 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:34.740429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:34.740554 (+ 125us) server_negotiation.cc:407] Connection header received
0807 02:31:34.740684 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:34.740686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:34.740714 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:34.740829 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:34.741546 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:34.741549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:34.741551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:34.741562 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:34.741587 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:34.741597 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:34.741671 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:34.741788 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 02:31:34.741822 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 02:31:34.743271 18338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } }
I20260807 02:31:34.743491 18338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:34.743566 18338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e03ebc5b9d4fcd9eb02506a9d0ddc4, State: Initialized, Role: FOLLOWER
I20260807 02:31:34.744045 18338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } }
I20260807 02:31:34.744215 18338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:34.744270 18338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:34.744367 18338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:34.745440 18338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } }
I20260807 02:31:34.745810 18338 leader_election.cc:302] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e03ebc5b9d4fcd9eb02506a9d0ddc4; no voters:
I20260807 02:31:34.746155 18338 leader_election.cc:288] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:34.746260 18356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:34.746515 18356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [term 1 LEADER]: Becoming Leader. State: Replica: 86e03ebc5b9d4fcd9eb02506a9d0ddc4, State: Running, Role: LEADER
I20260807 02:31:34.746908 18356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } }
I20260807 02:31:34.747498 18338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:34.748946 18357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } } }
I20260807 02:31:34.749073 18357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:34.749419 18359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e03ebc5b9d4fcd9eb02506a9d0ddc4. Latest consensus state: current_term: 1 leader_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e03ebc5b9d4fcd9eb02506a9d0ddc4" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35503 } } }
I20260807 02:31:34.749580 18359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4 [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:34.752326 18373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:34.752393 18373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:34.752475 18367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:34.753326 18367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:34.758464 18367 catalog_manager.cc:1347] Generated new cluster ID: 5aa3b9fd3ac046a491bc0cd3b70d18a6
I20260807 02:31:34.758530 18367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:34.767984 18367 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:34.768293 18367 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:34.772533 18367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e03ebc5b9d4fcd9eb02506a9d0ddc4: Generated new TSK 0
I20260807 02:31:34.773213 18367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:34.779515 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18227
2026-08-07T02:31:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:34 dist-test-slave-g170 krb5kdc[18380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:34 dist-test-slave-g170 krb5kdc[18380](info): set up 2 sockets
Aug 07 02:31:34 dist-test-slave-g170 krb5kdc[18380](info): commencing operation
krb5kdc: starting...
W20260807 02:31:36.038718 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:36.180578 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46685
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:46685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:36.301936 18404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:36.302232 18404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:36.302314 18404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:36.306222 18404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:36.306301 18404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:36.306341 18404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:36.306397 18404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:36.306434 18404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:36.311745 18404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:46685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:36.312857 18404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:36.313905 18404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 02:31:36.322955 18404 server_base.cc:1034] running on GCE node
W20260807 02:31:36.322894 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:36.322906 18416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:36.322885 18419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:36.323793 18404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:36.324983 18404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:36.326206 18404 hybrid_clock.cc:648] HybridClock initialized: now 1786069896326180 us; error 45 us; skew 500 ppm
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069896, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:36.334306 18404 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:36.335203 18404 webserver.cc:460] Webserver started at http://127.17.53.62:37183/ using document root <none> and password file <none>
I20260807 02:31:36.335788 18404 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:36.335882 18404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:36.336123 18404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:36.338318 18404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "a008883061e34a228acfc8d4bc838fc6"
format_stamp: "Formatted at 2026-08-07 02:31:36 on dist-test-slave-g170"
server_key: "7d64daa3c7f272c509cd4f78c628424e"
server_key_iv: "909489dc6ac3d75c3127f082682eac55"
server_key_version: "encryptionkey@0"
I20260807 02:31:36.338913 18404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "a008883061e34a228acfc8d4bc838fc6"
format_stamp: "Formatted at 2026-08-07 02:31:36 on dist-test-slave-g170"
server_key: "7d64daa3c7f272c509cd4f78c628424e"
server_key_iv: "909489dc6ac3d75c3127f082682eac55"
server_key_version: "encryptionkey@0"
I20260807 02:31:36.342850 18404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:36.345387 18429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:36.346629 18404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:36.346817 18404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "a008883061e34a228acfc8d4bc838fc6"
format_stamp: "Formatted at 2026-08-07 02:31:36 on dist-test-slave-g170"
server_key: "7d64daa3c7f272c509cd4f78c628424e"
server_key_iv: "909489dc6ac3d75c3127f082682eac55"
server_key_version: "encryptionkey@0"
I20260807 02:31:36.346936 18404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:36.362203 18404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:36.365315 18404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:36.365525 18404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:36.373224 18404 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:46685
I20260807 02:31:36.373240 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:46685 every 8 connection(s)
I20260807 02:31:36.374198 18404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:36.377138 18510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:36.379060 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18404
I20260807 02:31:36.379200 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:36.379499 17620 external_mini_cluster.cc:1428] Setting key 574ef089edd858ef23e76552ec026864
I20260807 02:31:36.384241 18510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6: Bootstrap starting.
I20260807 02:31:36.386602 18510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:36.387676 18510 log.cc:824] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6: Log is configured to *not* fsync() on all Append() calls
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:36.390287 18510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6: No bootstrap required, opened a new log
I20260807 02:31:36.394569 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:36.380899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59169 (local address 127.17.53.62:46685)
0807 02:31:36.381305 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:36.381322 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:36.381358 (+ 36us) server_negotiation.cc:407] Connection header received
0807 02:31:36.381995 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:36.382010 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:36.382344 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:36.382692 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:36.383535 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:36.384563 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:36.385500 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:36.385968 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:36.386144 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:36.389511 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:36.389518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:36.389534 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:36.389566 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:36.391515 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:36.392041 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:36.392046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:36.392052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:36.392178 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:36.392517 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:36.392520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:36.392521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:36.393006 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:36.393165 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:36.393474 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 02:31:36.393838 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":106,"threads_started":1}
I20260807 02:31:36.394856 18510 raft_consensus.cc:348] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } }
I20260807 02:31:36.395117 18510 raft_consensus.cc:374] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:36.395206 18510 raft_consensus.cc:729] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a008883061e34a228acfc8d4bc838fc6, State: Initialized, Role: FOLLOWER
I20260807 02:31:36.395654 18510 consensus_queue.cc:260] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } }
I20260807 02:31:36.395813 18510 raft_consensus.cc:388] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:36.395902 18510 raft_consensus.cc:482] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:36.396027 18510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:36.397092 18510 raft_consensus.cc:504] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } }
I20260807 02:31:36.397449 18510 leader_election.cc:302] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a008883061e34a228acfc8d4bc838fc6; no voters:
I20260807 02:31:36.397817 18510 leader_election.cc:288] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:36.397812 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:36.397912 18519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:36.398124 18519 raft_consensus.cc:686] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [term 1 LEADER]: Becoming Leader. State: Replica: a008883061e34a228acfc8d4bc838fc6, State: Running, Role: LEADER
I20260807 02:31:36.398536 18519 consensus_queue.cc:237] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } }
I20260807 02:31:36.399122 18510 sys_catalog.cc:564] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:36.399801 18519 sys_catalog.cc:455] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a008883061e34a228acfc8d4bc838fc6. Latest consensus state: current_term: 1 leader_uuid: "a008883061e34a228acfc8d4bc838fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } } }
I20260807 02:31:36.400002 18519 sys_catalog.cc:458] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:36.400259 18522 sys_catalog.cc:455] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a008883061e34a228acfc8d4bc838fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a008883061e34a228acfc8d4bc838fc6" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46685 } } }
I20260807 02:31:36.400365 18522 sys_catalog.cc:458] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:36.400367 18537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:36.403748 18537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:36.409955 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:36.400915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42577 (local address 127.17.53.62:46685)
0807 02:31:36.401016 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:36.401020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:36.401648 (+ 628us) server_negotiation.cc:407] Connection header received
0807 02:31:36.401777 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:36.401780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:36.401827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:36.401905 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:36.402729 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:36.403385 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:36.404436 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:36.404794 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:36.404899 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:36.406471 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:36.406475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:36.406478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:36.406502 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:36.408113 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:36.408609 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:36.408612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:36.408613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:36.408692 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:36.409041 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:36.409044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:36.409045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:36.409283 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:36.409416 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:36.409694 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 02:31:36.409814 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 02:31:36.410521 18537 catalog_manager.cc:1347] Generated new cluster ID: bd66a3d18e17459583e76ebc2f551540
I20260807 02:31:36.410624 18537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:36.423146 18537 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:36.424365 18537 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:36.430105 18537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a008883061e34a228acfc8d4bc838fc6: Generated new TSK 0
I20260807 02:31:36.430899 18537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:36.434933 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18404
2026-08-07T02:31:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18558](info): set up 2 sockets
Aug 07 02:31:36 dist-test-slave-g170 krb5kdc[18558](info): commencing operation
krb5kdc: starting...
W20260807 02:31:37.690239 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:37 dist-test-slave-g170 krb5kdc[18558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:37.833204 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46519
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:46519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:37.951293 18580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:37.951622 18580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:37.951707 18580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:37.955746 18580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:37.955844 18580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:37.955881 18580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:37.955933 18580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:37.955973 18580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:37.961354 18580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:46519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:46519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:37.962483 18580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:37.963570 18580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:37.972957 18589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:37.973001 18592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:37.973206 18580 server_base.cc:1034] running on GCE node
W20260807 02:31:37.973026 18594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:37.973932 18580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:37.975153 18580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:37.976342 18580 hybrid_clock.cc:648] HybridClock initialized: now 1786069897976311 us; error 45 us; skew 500 ppm
Aug 07 02:31:37 dist-test-slave-g170 krb5kdc[18558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069897, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:37.979185 18580 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:37.980041 18580 webserver.cc:460] Webserver started at http://127.17.53.62:34493/ using document root <none> and password file <none>
I20260807 02:31:37.980611 18580 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:37.980669 18580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:37.980965 18580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:37.983013 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "351c0e2486274c43b74bd0756a6190b1"
format_stamp: "Formatted at 2026-08-07 02:31:37 on dist-test-slave-g170"
server_key: "c751f266d9a3d9568f31a0c15c19dbb3"
server_key_iv: "4f711934e0be863603e6a2470cb80c2c"
server_key_version: "encryptionkey@0"
I20260807 02:31:37.983521 18580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "351c0e2486274c43b74bd0756a6190b1"
format_stamp: "Formatted at 2026-08-07 02:31:37 on dist-test-slave-g170"
server_key: "c751f266d9a3d9568f31a0c15c19dbb3"
server_key_iv: "4f711934e0be863603e6a2470cb80c2c"
server_key_version: "encryptionkey@0"
I20260807 02:31:37.987344 18580 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 02:31:37.989756 18601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:37.991195 18580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 02:31:37.991313 18580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "351c0e2486274c43b74bd0756a6190b1"
format_stamp: "Formatted at 2026-08-07 02:31:37 on dist-test-slave-g170"
server_key: "c751f266d9a3d9568f31a0c15c19dbb3"
server_key_iv: "4f711934e0be863603e6a2470cb80c2c"
server_key_version: "encryptionkey@0"
I20260807 02:31:37.991407 18580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:38.010116 18580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:38.013674 18580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:38.013876 18580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:38.021632 18580 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:46519
I20260807 02:31:38.021656 18681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:46519 every 8 connection(s)
I20260807 02:31:38.022742 18580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:38.025923 18687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:38.031303 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18580
I20260807 02:31:38.031473 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:38.031807 17620 external_mini_cluster.cc:1428] Setting key ed7bd84cf389f37ca51b8aeb7633f199
I20260807 02:31:38.032459 18687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1: Bootstrap starting.
I20260807 02:31:38.035138 18687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:38.035960 18687 log.cc:824] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:38.038008 18687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1: No bootstrap required, opened a new log
I20260807 02:31:38.040875 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:38.033354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43091 (local address 127.17.53.62:46519)
0807 02:31:38.033833 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:38.033855 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:38.033892 (+ 37us) server_negotiation.cc:407] Connection header received
0807 02:31:38.034552 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:38.034570 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:38.034911 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:38.035259 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:38.035686 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:38.036710 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:38.037651 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:38.038155 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:38.038331 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:38.039183 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:38.039189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:38.039203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:38.039574 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:38.039625 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:38.039649 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:38.039751 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:38.039913 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 02:31:38.040116 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":161,"threads_started":1}
I20260807 02:31:38.042217 18687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } }
I20260807 02:31:38.042517 18687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:38.042582 18687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351c0e2486274c43b74bd0756a6190b1, State: Initialized, Role: FOLLOWER
I20260807 02:31:38.043077 18687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } }
I20260807 02:31:38.043246 18687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:38.043344 18687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:38.043476 18687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:38.043948 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:38.044541 18687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } }
I20260807 02:31:38.044937 18687 leader_election.cc:302] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 351c0e2486274c43b74bd0756a6190b1; no voters:
I20260807 02:31:38.045341 18687 leader_election.cc:288] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:38.045411 18697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:38.045646 18697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [term 1 LEADER]: Becoming Leader. State: Replica: 351c0e2486274c43b74bd0756a6190b1, State: Running, Role: LEADER
I20260807 02:31:38.045987 18697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } }
I20260807 02:31:38.046754 18687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:38.047411 18701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "351c0e2486274c43b74bd0756a6190b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } } }
I20260807 02:31:38.047554 18701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:38.047981 18700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 351c0e2486274c43b74bd0756a6190b1. Latest consensus state: current_term: 1 leader_uuid: "351c0e2486274c43b74bd0756a6190b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351c0e2486274c43b74bd0756a6190b1" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 46519 } } }
I20260807 02:31:38.048094 18700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:38.049338 18708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:38.052380 18708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:38.053691 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:38.046070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33655 (local address 127.17.53.62:46519)
0807 02:31:38.046216 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:38.046220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:38.049453 (+ 3233us) server_negotiation.cc:407] Connection header received
0807 02:31:38.049610 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:38.049614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:38.049663 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:38.049754 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:38.050030 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:38.050589 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:38.051510 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:38.051924 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:38.052046 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:38.053163 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:38.053167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:38.053170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:38.053184 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:38.053217 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:38.053229 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:38.053335 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:38.053436 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 02:31:38.053487 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:31:38.058462 18708 catalog_manager.cc:1347] Generated new cluster ID: caf7243f88534475900ab7b37edbeca2
I20260807 02:31:38.058529 18708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:38.087442 18708 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:38.088421 18708 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:38.092844 18708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 351c0e2486274c43b74bd0756a6190b1: Generated new TSK 0
I20260807 02:31:38.093537 18708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:38.098857 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18580
2026-08-07T02:31:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:38 dist-test-slave-g170 krb5kdc[18736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:38 dist-test-slave-g170 krb5kdc[18736](info): set up 2 sockets
Aug 07 02:31:38 dist-test-slave-g170 krb5kdc[18736](info): commencing operation
krb5kdc: starting...
W20260807 02:31:39.417755 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:39 dist-test-slave-g170 krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:39.551216 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:42483
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:45323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:42483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:39.665622 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:39.666092 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:39.666261 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:39.670449 18757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:39.670509 18757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:39.670529 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:39.670545 18757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:39.670600 18757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:39.675983 18757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:45323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:42483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:42483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:39.677201 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:39.678285 18757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:39.687117 18768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:39.687144 18766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:39.687304 18757 server_base.cc:1034] running on GCE node
W20260807 02:31:39.687497 18765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:39.688179 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:39.689536 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:39.690758 18757 hybrid_clock.cc:648] HybridClock initialized: now 1786069899690723 us; error 47 us; skew 500 ppm
Aug 07 02:31:39 dist-test-slave-g170 krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069899, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:39.693924 18757 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:39.695024 18757 webserver.cc:460] Webserver started at http://127.17.53.62:40559/ using document root <none> and password file <none>
I20260807 02:31:39.695680 18757 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:39.695787 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:39.696062 18757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:39.698529 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "7735ed8ad19341adb61f189316ae4fe5"
format_stamp: "Formatted at 2026-08-07 02:31:39 on dist-test-slave-g170"
server_key: "8f8e2f5396602507d1cf643cd3caf960"
server_key_iv: "bc03733f4125e382c2f33cf442811739"
server_key_version: "encryptionkey@0"
I20260807 02:31:39.699186 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "7735ed8ad19341adb61f189316ae4fe5"
format_stamp: "Formatted at 2026-08-07 02:31:39 on dist-test-slave-g170"
server_key: "8f8e2f5396602507d1cf643cd3caf960"
server_key_iv: "bc03733f4125e382c2f33cf442811739"
server_key_version: "encryptionkey@0"
I20260807 02:31:39.703842 18757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:39.707027 18776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:39.708691 18757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 02:31:39.708915 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "7735ed8ad19341adb61f189316ae4fe5"
format_stamp: "Formatted at 2026-08-07 02:31:39 on dist-test-slave-g170"
server_key: "8f8e2f5396602507d1cf643cd3caf960"
server_key_iv: "bc03733f4125e382c2f33cf442811739"
server_key_version: "encryptionkey@0"
I20260807 02:31:39.709066 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:39.714872 18757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:39.718500 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:39.718724 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:39.726511 18757 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:42483
I20260807 02:31:39.726536 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:42483 every 8 connection(s)
I20260807 02:31:39.727593 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:39.727828 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18757
I20260807 02:31:39.727967 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:39.728341 17620 external_mini_cluster.cc:1428] Setting key a5a40579bc4a0f2dfbe54e16f9e0d34a
I20260807 02:31:39.731866 18852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:39.734930 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:39.729975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43699 (local address 127.17.53.62:42483)
0807 02:31:39.730695 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:39.730715 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:39.730765 (+ 50us) server_negotiation.cc:407] Connection header received
0807 02:31:39.731700 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:39.731724 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:39.732122 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:39.732523 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:39.733341 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:39.733347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:39.733363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:39.733377 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:39.733444 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:39.733470 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:39.733623 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:39.733811 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 02:31:39.733970 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":142,"threads_started":1}
I20260807 02:31:39.738132 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:39.739678 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5: Bootstrap starting.
I20260807 02:31:39.742122 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:39.740378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39983 (local address 127.17.53.62:42483)
0807 02:31:39.740520 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:39.740525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:39.740706 (+ 181us) server_negotiation.cc:407] Connection header received
0807 02:31:39.740874 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:39.740877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:39.740921 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:39.741031 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:39.741736 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:39.741741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:39.741743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 02:31:39.741759 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:39.741792 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 02:31:39.741807 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:39.741915 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:39.741949 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 02:31:39.741989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 02:31:39.742847 18852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:39.743796 18852 log.cc:824] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:39.746234 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5: No bootstrap required, opened a new log
I20260807 02:31:39.750099 18852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } }
I20260807 02:31:39.750398 18852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 02:31:39.750479 18852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7735ed8ad19341adb61f189316ae4fe5, State: Initialized, Role: FOLLOWER
I20260807 02:31:39.750980 18852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } }
I20260807 02:31:39.751138 18852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 02:31:39.751216 18852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 02:31:39.751338 18852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:39.752362 18852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } }
I20260807 02:31:39.752735 18852 leader_election.cc:302] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7735ed8ad19341adb61f189316ae4fe5; no voters:
I20260807 02:31:39.753155 18852 leader_election.cc:288] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 02:31:39.753327 18868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:39.753561 18868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [term 1 LEADER]: Becoming Leader. State: Replica: 7735ed8ad19341adb61f189316ae4fe5, State: Running, Role: LEADER
I20260807 02:31:39.753911 18868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } }
I20260807 02:31:39.754530 18852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:39.755857 18870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7735ed8ad19341adb61f189316ae4fe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } } }
I20260807 02:31:39.755910 18871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7735ed8ad19341adb61f189316ae4fe5. Latest consensus state: current_term: 1 leader_uuid: "7735ed8ad19341adb61f189316ae4fe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7735ed8ad19341adb61f189316ae4fe5" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 42483 } } }
I20260807 02:31:39.755988 18870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:39.756021 18871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:39.756415 18881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:39.759711 18881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:39.765592 18881 catalog_manager.cc:1347] Generated new cluster ID: 8e812f71d1ae435d860d131c0de47e78
I20260807 02:31:39.765682 18881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:39.777613 18881 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:39.777943 18881 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:39.787403 18881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7735ed8ad19341adb61f189316ae4fe5: Generated new TSK 0
I20260807 02:31:39.788130 18881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 02:31:39.799503 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18757
2026-08-07T02:31:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1702 ms)
[----------] 18 tests from AuthTokenIssuingTest (30243 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:39 dist-test-slave-g170 krb5kdc[18898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:39 dist-test-slave-g170 krb5kdc[18898](info): set up 2 sockets
Aug 07 02:31:39 dist-test-slave-g170 krb5kdc[18898](info): commencing operation
krb5kdc: starting...
W20260807 02:31:41.099882 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:41.235496 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41935
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:41.350656 18919 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:41.350951 18919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:41.350976 18919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:41.351037 18919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:41.355095 18919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:41.355147 18919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:41.355166 18919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:41.355182 18919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:41.355197 18919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:41.360332 18919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:41935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:41.361441 18919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:41.362480 18919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:41.370811 18930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.370769 18926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.370899 18924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:41.371188 18919 server_base.cc:1034] running on GCE node
I20260807 02:31:41.371752 18919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:41.373057 18919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:41.374239 18919 hybrid_clock.cc:648] HybridClock initialized: now 1786069901374211 us; error 39 us; skew 500 ppm
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:41.381808 18919 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:41.382697 18919 webserver.cc:460] Webserver started at http://127.17.53.62:43549/ using document root <none> and password file <none>
I20260807 02:31:41.383272 18919 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:41.383364 18919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:41.383600 18919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:41.386149 18919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "80a5c73942aa474a9cdb80fd75ff0af2"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "fa7f8d3ef9300c463710b22000406c98"
server_key_iv: "147aff47d7475e2eb8927098ca10763b"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.386749 18919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "80a5c73942aa474a9cdb80fd75ff0af2"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "fa7f8d3ef9300c463710b22000406c98"
server_key_iv: "147aff47d7475e2eb8927098ca10763b"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.390623 18919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:41.393148 18941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:41.394474 18919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:41.394631 18919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "80a5c73942aa474a9cdb80fd75ff0af2"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "fa7f8d3ef9300c463710b22000406c98"
server_key_iv: "147aff47d7475e2eb8927098ca10763b"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.394740 18919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:41.416121 18919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:41.419574 18919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:41.419791 18919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:41.427137 18919 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:41935
I20260807 02:31:41.427425 19024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:41935 every 8 connection(s)
I20260807 02:31:41.428130 18919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:41.430996 19026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:41.432389 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 18919
I20260807 02:31:41.432521 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:41.432875 17620 external_mini_cluster.cc:1428] Setting key d055a714d31a266c1d3a980a2a6a46b2
I20260807 02:31:41.436014 19026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.53.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.61@KRBTEST.COM" created.
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:41.452432 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.438290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41935 (local address 127.0.0.1:60942)
0807 02:31:41.438778 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.438830 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.439755 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.440572 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.440577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.440973 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.441241 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.441255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.442801 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.442804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.443568 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.443578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.444304 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.444307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.444402 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.445144 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.445161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.447023 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.449450 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.449458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.449466 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.449708 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.450155 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.450157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.450159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.450293 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.451148 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.451153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.451282 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.451423 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.451654 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":138,"threads_started":1}
I20260807 02:31:41.452438 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.439205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.17.53.62:41935)
0807 02:31:41.439552 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.439565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.439616 (+ 51us) server_negotiation.cc:407] Connection header received
0807 02:31:41.440087 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.440094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.440344 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.440560 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.441406 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.442684 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.443672 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.444125 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.444248 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.447177 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.447187 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.447201 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.447232 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.449334 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.449875 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.449884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.449888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.450041 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.450391 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.450398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.450403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.451013 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.451173 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.451554 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.451728 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260807 02:31:41.455061 19026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.455838 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.455570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43700)
0807 02:31:41.455682 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.455763 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 02:31:41.456077 18945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
I20260807 02:31:41.457021 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.456822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43716)
0807 02:31:41.456923 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.456968 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 02:31:41.457155 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:36299: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
I20260807 02:31:41.463341 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:36299
--webserver_interface=127.17.53.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:31:41.506418 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } attempt: 1
I20260807 02:31:41.507332 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.507023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43724)
0807 02:31:41.507187 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.507251 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 02:31:41.508211 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.507974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43738)
0807 02:31:41.508091 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.508141 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 02:31:41.508396 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:36299: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
I20260807 02:31:41.576548 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } attempt: 2
I20260807 02:31:41.577466 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.577163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43754)
0807 02:31:41.577326 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.577390 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 02:31:41.578222 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.577992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43764)
0807 02:31:41.578114 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.578165 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 02:31:41.578352 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:36299: Network error: Client connection negotiation failed: client connection to 127.17.53.61:36299: connect: Connection refused (error 111)
W20260807 02:31:41.584012 19036 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:41.584322 19036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:41.584385 19036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:41.584465 19036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:41.588861 19036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:41.588937 19036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:41.588982 19036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:41.589013 19036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:41.589052 19036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:41.594138 19036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:36299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.53.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:41.595168 19036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:41.596184 19036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:41.604919 19043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.604928 19042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.604928 19045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:41.605268 19036 server_base.cc:1034] running on GCE node
I20260807 02:31:41.605832 19036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:41.606868 19036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:41.608074 19036 hybrid_clock.cc:648] HybridClock initialized: now 1786069901608048 us; error 39 us; skew 500 ppm
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:41.610940 19036 init.cc:377] Logged in from keytab as kudu/127.17.53.61@KRBTEST.COM (short username kudu)
I20260807 02:31:41.611824 19036 webserver.cc:460] Webserver started at http://127.17.53.61:33671/ using document root <none> and password file <none>
I20260807 02:31:41.612382 19036 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:41.612473 19036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:41.612705 19036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:41.614724 19036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data/instance:
uuid: "b005fbad47064eec92765f744bbd2bfc"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "7ef9e20b67be2ce1883785e365684d97"
server_key_iv: "e1cb6bb8b3cdd127cc56b44df271a9c0"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.615299 19036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal/instance:
uuid: "b005fbad47064eec92765f744bbd2bfc"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "7ef9e20b67be2ce1883785e365684d97"
server_key_iv: "e1cb6bb8b3cdd127cc56b44df271a9c0"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.619115 19036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 02:31:41.621603 19052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:41.622854 19036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:31:41.623009 19036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "b005fbad47064eec92765f744bbd2bfc"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "7ef9e20b67be2ce1883785e365684d97"
server_key_iv: "e1cb6bb8b3cdd127cc56b44df271a9c0"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.623131 19036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:41.635329 19036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:41.638474 19036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:41.638684 19036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:41.646469 19036 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.61:36299
I20260807 02:31:41.646783 19130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.61:36299 every 8 connection(s)
I20260807 02:31:41.647511 19036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
I20260807 02:31:41.650178 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19036
I20260807 02:31:41.650362 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-1/wal/instance
I20260807 02:31:41.650696 17620 external_mini_cluster.cc:1428] Setting key 54d3c8214d9406cba21dafc94f4267bd
I20260807 02:31:41.650702 19131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:41.654536 19131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.53.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.60@KRBTEST.COM" created.
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:41.668835 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.656469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.17.53.62:41935)
0807 02:31:41.656659 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.656667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.657515 (+ 848us) server_negotiation.cc:407] Connection header received
0807 02:31:41.658452 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.658456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.658510 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.658628 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.659790 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.660357 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.661641 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.661942 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.662032 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.664625 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.664632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.664638 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.664673 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.666589 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.667099 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.667105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.667108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.667207 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.667532 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.667538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.667542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.667835 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.667955 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.668442 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.668593 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 02:31:41.669560 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.656854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41935 (local address 127.0.0.1:60944)
0807 02:31:41.657337 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.657389 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.658233 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.658769 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.658776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.659090 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.659629 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.659643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.660471 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.660473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.661528 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.661539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.662112 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.662119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.662371 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.663015 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.663032 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.664460 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.666730 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.666736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.666751 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.667000 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.667302 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.667304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.667305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.667431 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.667942 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.667947 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.668170 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.668342 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.668583 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":142,"threads_started":1}
I20260807 02:31:41.669536 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } attempt: 3
I20260807 02:31:41.671032 19131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
I20260807 02:31:41.680213 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:43601
--webserver_interface=127.17.53.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:46731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:31:41.681393 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.671677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43788)
0807 02:31:41.671794 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.671810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.671879 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.672309 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.672312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.672475 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.672578 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.672584 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.673489 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.673496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.674090 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.674099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.674809 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.674955 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.675583 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.675616 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.677382 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.679752 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.679755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.679757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.679988 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.680303 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.680312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.680403 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.680888 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.680891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.680979 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.681089 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.681220 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":20}
I20260807 02:31:41.681517 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.671812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43788 (local address 127.17.53.61:36299)
0807 02:31:41.672096 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.672102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.672113 (+ 11us) server_negotiation.cc:407] Connection header received
0807 02:31:41.672141 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.672142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.672174 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.672248 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.672695 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.673382 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674199 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.674695 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674852 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.677569 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.677575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.677580 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.677610 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.679636 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680088 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.680095 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.680096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.680186 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680498 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.680503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.680505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.680747 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.680917 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.681192 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.681327 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":230,"spinlock_wait_cycles":4992,"thread_start_us":146,"threads_started":1}
I20260807 02:31:41.681771 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.669955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43774)
0807 02:31:41.670241 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.670256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.670333 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.671299 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.671302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.671487 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.671624 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.671632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.672962 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.672965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.674121 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.674131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.674822 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.674950 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.675509 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.675528 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.677083 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.680036 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.680043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.680245 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.680558 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680564 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.680569 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.680671 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.681219 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.681231 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.681347 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.681440 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.681615 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260807 02:31:41.682585 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.670499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43774 (local address 127.17.53.61:36299)
0807 02:31:41.670865 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.670876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.670899 (+ 23us) server_negotiation.cc:407] Connection header received
0807 02:31:41.670941 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.670946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.671148 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.671298 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.671749 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.672848 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674235 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.674688 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.674836 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.677259 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.677268 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.677279 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.677300 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.679909 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680349 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.680354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.680357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.680446 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.680823 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.680828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.680833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.681083 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.682073 (+ 990us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.682157 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.682308 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":243,"spinlock_wait_cycles":3456,"thread_start_us":139,"threads_started":1}
I20260807 02:31:41.683471 19026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.683976 19131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.684214 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.683944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45334)
0807 02:31:41.684084 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.684143 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 02:31:41.684672 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.684381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45336)
0807 02:31:41.684503 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.684589 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 02:31:41.684942 19053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
I20260807 02:31:41.685061 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.684796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45346)
0807 02:31:41.684930 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.684977 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 02:31:41.685209 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
I20260807 02:31:41.686101 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.685810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45356)
0807 02:31:41.685947 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.686017 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 02:31:41.686273 19131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
I20260807 02:31:41.716380 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 1
I20260807 02:31:41.717329 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.717003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45362)
0807 02:31:41.717158 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.717228 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 02:31:41.718189 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.717947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45364)
0807 02:31:41.718085 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.718134 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 02:31:41.718371 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
I20260807 02:31:41.735556 19131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 1
I20260807 02:31:41.736570 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.736180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45376)
0807 02:31:41.736371 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.736445 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 02:31:41.737612 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.737295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45378)
0807 02:31:41.737445 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.737517 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 02:31:41.737819 19131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
I20260807 02:31:41.793560 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 2
I20260807 02:31:41.794540 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.794194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45380)
0807 02:31:41.794386 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.794454 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 02:31:41.795398 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.795169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45394)
0807 02:31:41.795305 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.795350 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 02:31:41.795540 19026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
W20260807 02:31:41.803640 19140 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:41.803966 19140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:41.804026 19140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:41.804102 19140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 02:31:41.805996 19131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 2
I20260807 02:31:41.806866 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.806534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45404)
0807 02:31:41.806693 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.806759 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 02:31:41.807703 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.807479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45410)
0807 02:31:41.807589 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.807640 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 02:31:41.807878 19131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:43601: Network error: Client connection negotiation failed: client connection to 127.17.53.60:43601: connect: Connection refused (error 111)
W20260807 02:31:41.808917 19140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:41.808964 19140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:41.808984 19140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:41.809016 19140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:41.809051 19140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:41.814409 19140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:46731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:41935,127.17.53.61:36299,127.17.53.60:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.53.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:41.815474 19140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:41.816505 19140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:41.825045 19151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.825037 19146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:41.825038 19149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:41.825333 19140 server_base.cc:1034] running on GCE node
I20260807 02:31:41.825911 19140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:41.827096 19140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:41.828295 19140 hybrid_clock.cc:648] HybridClock initialized: now 1786069901828278 us; error 27 us; skew 500 ppm
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:41.831101 19140 init.cc:377] Logged in from keytab as kudu/127.17.53.60@KRBTEST.COM (short username kudu)
I20260807 02:31:41.831984 19140 webserver.cc:460] Webserver started at http://127.17.53.60:32849/ using document root <none> and password file <none>
I20260807 02:31:41.832536 19140 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:41.832595 19140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:41.832883 19140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:41.834895 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data/instance:
uuid: "fe92f9f9d1f14391910212af26372959"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "d945b7eeeca2fde437173966d94ed409"
server_key_iv: "3449b1e165bf84798f3a93ef00b0db4d"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.835461 19140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal/instance:
uuid: "fe92f9f9d1f14391910212af26372959"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "d945b7eeeca2fde437173966d94ed409"
server_key_iv: "3449b1e165bf84798f3a93ef00b0db4d"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.839195 19140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 02:31:41.841724 19164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:41.842993 19140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 02:31:41.843127 19140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "fe92f9f9d1f14391910212af26372959"
format_stamp: "Formatted at 2026-08-07 02:31:41 on dist-test-slave-g170"
server_key: "d945b7eeeca2fde437173966d94ed409"
server_key_iv: "3449b1e165bf84798f3a93ef00b0db4d"
server_key_version: "encryptionkey@0"
I20260807 02:31:41.843230 19140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:41.860214 19140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:41.863554 19140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:41.863762 19140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:41.871274 19140 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.60:43601
I20260807 02:31:41.871572 19237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.60:43601 every 8 connection(s)
I20260807 02:31:41.872289 19140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
I20260807 02:31:41.875196 19242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:41.879076 19242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.879838 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19140
I20260807 02:31:41.879953 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786069602236550-17620-0/minicluster-data/master-2/wal/instance
I20260807 02:31:41.880220 17620 external_mini_cluster.cc:1428] Setting key f36f9dc4c688d7ce1d3d134cf364fe23
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:31:41.894567 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.881084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.17.53.62:41935)
0807 02:31:41.881231 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.881235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.882169 (+ 934us) server_negotiation.cc:407] Connection header received
0807 02:31:41.883057 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.883060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.883111 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.883189 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.884591 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.885382 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.886795 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.887139 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.887268 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.890431 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.890434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.890436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.890459 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.892118 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.892962 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.892965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.892967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.893042 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.893484 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.893487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.893488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.893667 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.893782 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.894275 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.894394 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:31:41.894821 19026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 3
I20260807 02:31:41.895704 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.881448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:41935 (local address 127.0.0.1:60958)
0807 02:31:41.882007 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.882060 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.882820 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.883801 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.883808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.884239 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.884442 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.884452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.885501 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.885504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.886675 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.886688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.887309 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.887312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.887470 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.888462 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.888483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.890283 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.892242 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.892252 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.892260 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.892842 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.893149 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.893156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.893162 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.893373 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.893791 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.893800 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.894038 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.894223 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.894521 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":25,"spinlock_wait_cycles":43648,"thread_start_us":158,"threads_started":1}
I20260807 02:31:41.895713 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.881448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.17.53.60:43601)
0807 02:31:41.881977 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.881995 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.882037 (+ 42us) server_negotiation.cc:407] Connection header received
0807 02:31:41.882819 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.882843 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.883081 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.883265 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.884166 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.885591 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.886776 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.887340 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.887466 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.891299 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.891305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.891315 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.891341 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.893475 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.893953 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.893960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.893962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.894065 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.894389 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.894395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.894397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.894719 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.894966 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.895209 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.895374 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":417,"thread_start_us":156,"threads_started":1}
I20260807 02:31:41.897269 19242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.899061 19131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } attempt: 3
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:31:41.905973 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.895951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45428 (local address 127.17.53.60:43601)
0807 02:31:41.896205 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.896209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.896224 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:31:41.896260 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.896271 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.896312 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.896392 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.896867 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.897545 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.898757 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.899118 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.900423 (+ 1305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.901786 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.901792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.901796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.901830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.903722 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.904338 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.904342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.904344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.904438 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.904872 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.904875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.904878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.905102 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.905226 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.905582 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.905753 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":181}
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:31:41.906522 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.895311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45428)
0807 02:31:41.895528 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.895546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.895651 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.896499 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.896502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.896639 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.896716 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.896720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.897707 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.897715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.898638 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.898654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.899244 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.899248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.899334 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.899799 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.899813 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.901615 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.903865 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.903869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.903871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.904180 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.904562 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.904566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.904568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.904692 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.905236 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.905242 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.905356 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.905470 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.906335 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:41.910035 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.898161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43804 (local address 127.17.53.61:36299)
0807 02:31:41.898289 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.898293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.898307 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:31:41.898351 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.898354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.898409 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.898493 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.899016 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.899823 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.900888 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.901340 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.901470 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.904160 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.904164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.904166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.904191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.906884 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.907351 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.907356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.907357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.907438 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.908119 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.908124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.908127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.908982 (+ 855us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.909068 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.909703 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.909846 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 02:31:41.910735 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.897835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:36299 (local address 127.0.0.1:43804)
0807 02:31:41.898052 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.898068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.898164 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.898595 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.898597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.898783 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.898876 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.898882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.900006 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.900009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.900751 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.900763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.901471 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.901475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.901599 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.902288 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.902307 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.904034 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.906977 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.906983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.906987 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.907229 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.907559 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.907564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.907566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.907988 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.909150 (+ 1162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.909154 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.909246 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.910436 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.910577 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260807 02:31:41.910988 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.899492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45436)
0807 02:31:41.899701 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.899717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.899805 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.900713 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.900717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.900939 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.901028 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.901034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.902564 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.902569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.903424 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.903440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.904111 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.904117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.904208 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.904737 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.904754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.907237 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.909379 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.909386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.909391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.909663 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.909970 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.909973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.909975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.910057 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.910522 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.910525 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.910621 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.910701 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.910828 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 02:31:41.911168 19026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: Bootstrap starting.
I20260807 02:31:41.911549 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.900029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.17.53.60:43601)
0807 02:31:41.900408 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.900420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.900437 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:31:41.900482 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.900489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.900557 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.900677 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.901140 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.902421 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.903584 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.903965 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.904443 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.907388 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.907394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.907400 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.907425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.909245 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.909777 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.909779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.909781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.909866 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.910177 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.910181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.910184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.910404 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.911252 (+ 848us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.911338 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.911434 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"spinlock_wait_cycles":2944,"thread_start_us":156,"threads_started":1}
I20260807 02:31:41.911813 19242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:41.914139 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.898843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.17.53.62:41935)
0807 02:31:41.898980 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.898985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.899002 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:31:41.899406 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.899410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.899465 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.899548 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.902632 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.903500 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.904831 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.905204 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.905315 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.908326 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.908333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.908340 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.908376 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.910275 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.910871 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.910875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.910877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.910980 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.912532 (+ 1552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.912537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.912539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.912847 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.913057 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.913826 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.913954 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 02:31:41.914150 19131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: Bootstrap starting.
I20260807 02:31:41.914636 19026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:41.915786 19026 log.cc:824] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:41.916591 19131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:41.917371 19131 log.cc:824] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:41.918159 19026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: No bootstrap required, opened a new log
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
Aug 07 02:31:41 dist-test-slave-g170 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069901, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:31:41.922572 19026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.922900 19026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a5c73942aa474a9cdb80fd75ff0af2, State: Initialized, Role: FOLLOWER
I20260807 02:31:41.923010 19131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: No bootstrap required, opened a new log
I20260807 02:31:41.923303 19026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.923950 19259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.924091 19259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.925163 19026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:41.927546 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.912280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:43601 (local address 127.0.0.1:45440)
0807 02:31:41.912565 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:41.912580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:41.912655 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:41.918046 (+ 5391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:41.918048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:41.918208 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:41.918292 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.918298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.919081 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.919087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.919906 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:41.919919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.920627 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.920633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:41.920721 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.921200 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:41.921214 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:41.922517 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:41.925436 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.925439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.925442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.925740 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.926135 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:41.926140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:41.926143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:41.926267 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:41.926889 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:41.926892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:41.927000 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 02:31:41.927099 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 02:31:41.927282 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260807 02:31:41.927590 19131 raft_consensus.cc:348] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.927832 19131 raft_consensus.cc:729] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b005fbad47064eec92765f744bbd2bfc, State: Initialized, Role: FOLLOWER
I20260807 02:31:41.928700 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.912470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.17.53.60:43601)
0807 02:31:41.917798 (+ 5328us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.917802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.917815 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:31:41.917848 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.917851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.917889 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.917962 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.918410 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.918979 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.920049 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.920469 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.920590 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.922677 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.922682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.922685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.922716 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.925287 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.925881 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.925884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.925886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.925977 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.926415 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.926419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.926421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.926754 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.928265 (+ 1511us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.928368 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.928528 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5255}
I20260807 02:31:41.928857 19131 consensus_queue.cc:260] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.929646 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.915594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.17.53.61:36299)
0807 02:31:41.915698 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.915701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.915715 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:31:41.915836 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.915840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.915883 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.915961 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.916818 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.917589 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.918688 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.919041 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.919199 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.925385 (+ 6186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.925388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.925391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.925416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.927144 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.927690 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.927694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.927695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.927794 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.928132 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.928138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.928141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.928358 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.929000 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.929404 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.929515 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:31:41.932053 19242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: Bootstrap starting.
I20260807 02:31:41.933646 19131 sys_catalog.cc:564] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:41.934433 19242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:41.935091 19242 log.cc:824] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:41.931284 18919 master_runner.cc:186] Error getting master registration for 127.17.53.61:36299: OK, instance_id { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" instance_seqno: 1786069901644172 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:31:41.935950 18919 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:31:41.936513 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:41.937074 19242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: No bootstrap required, opened a new log
I20260807 02:31:41.937976 18980 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "80a5c73942aa474a9cdb80fd75ff0af2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60972
I20260807 02:31:41.938237 18980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 02:31:41.938408 18980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 0 FOLLOWER]: Advancing to term 1
W20260807 02:31:41.938669 19272 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:41.938737 19272 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:41.940057 18980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.940330 19274 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.940476 19274 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.941730 18980 leader_election.cc:288] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [CANDIDATE]: Term 1 election: Requested vote from peers b005fbad47064eec92765f744bbd2bfc (127.17.53.61:36299), fe92f9f9d1f14391910212af26372959 (127.17.53.60:43601)
I20260807 02:31:41.941745 19087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b005fbad47064eec92765f744bbd2bfc"
I20260807 02:31:41.941989 19087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:41.942022 19242 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.942242 19242 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe92f9f9d1f14391910212af26372959, State: Initialized, Role: FOLLOWER
I20260807 02:31:41.942554 19242 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.943643 19087 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a5c73942aa474a9cdb80fd75ff0af2 in term 1.
I20260807 02:31:41.943811 19242 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:41.944293 18945 leader_election.cc:302] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a5c73942aa474a9cdb80fd75ff0af2, b005fbad47064eec92765f744bbd2bfc; no voters:
I20260807 02:31:41.944551 19200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe92f9f9d1f14391910212af26372959"
I20260807 02:31:41.944885 19200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:41.945253 19259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:41.945464 19259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [term 1 LEADER]: Becoming Leader. State: Replica: 80a5c73942aa474a9cdb80fd75ff0af2, State: Running, Role: LEADER
I20260807 02:31:41.945816 19259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } }
I20260807 02:31:41.946403 19200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a5c73942aa474a9cdb80fd75ff0af2 in term 1.
I20260807 02:31:41.948042 19306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a5c73942aa474a9cdb80fd75ff0af2. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.948158 19306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: This master's current role is: LEADER
W20260807 02:31:41.950990 19322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:41.951072 19322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:41.951354 19325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:41.948128 19036 master_runner.cc:186] Error getting master registration for 127.17.53.60:43601: OK, instance_id { permanent_uuid: "fe92f9f9d1f14391910212af26372959" instance_seqno: 1786069901869170 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:31:41.951568 19036 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:31:41.952322 19325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:41.956945 19294 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.957088 19294 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 02:31:41.958101 19291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:41.958179 19291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:41.964159 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.953281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.17.53.61:36299)
0807 02:31:41.953402 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.953407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.954651 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 02:31:41.954862 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.954865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.954911 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.954963 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.956026 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.956812 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.958187 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.958462 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.958525 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.960134 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.960137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.960140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.960166 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.962105 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.962623 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.962627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.962629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.962728 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.963058 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.963061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.963063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.963365 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.963486 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.963859 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.963995 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 02:31:41.970832 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.954340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.17.53.60:43601)
0807 02:31:41.954445 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.954450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.961104 (+ 6654us) server_negotiation.cc:407] Connection header received
0807 02:31:41.961340 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.961344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.961404 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.961465 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.962373 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.963102 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.964459 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.964826 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.964902 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.966504 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.966508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.966511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.966537 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.968401 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.969197 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.969201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.969202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.969303 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.969664 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.969671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.969675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.969955 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.970084 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.970521 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.970641 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 02:31:41.973006 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:41.953960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.17.53.62:41935)
0807 02:31:41.954066 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:41.954070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:41.961030 (+ 6960us) server_negotiation.cc:407] Connection header received
0807 02:31:41.961222 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:41.961225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:41.961291 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:41.961345 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:41.962224 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.962996 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.966874 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:41.967243 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:41.967364 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:41.969188 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:41.969192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:41.969195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:41.969223 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:41.970949 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.971474 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.971478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.971480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.971565 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:41.972012 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:41.972016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:41.972017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:41.972235 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:41.972347 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:41.972705 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 02:31:41.972895 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 02:31:41.978255 19087 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [term 1 FOLLOWER]: Refusing update from remote peer 80a5c73942aa474a9cdb80fd75ff0af2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:41.978260 19200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [term 1 FOLLOWER]: Refusing update from remote peer 80a5c73942aa474a9cdb80fd75ff0af2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:41.979064 19306 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:41.979555 19259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:41.983297 19294 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a5c73942aa474a9cdb80fd75ff0af2. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.983287 19274 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a5c73942aa474a9cdb80fd75ff0af2. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.983419 19294 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.983443 19274 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.991600 19259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.991724 19259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:41.992115 19331 mvcc.cc:205] Tried to move back new op lower bound from 7315742318451613696 to 7315742318373507072. Current Snapshot: MvccSnapshot[applied={T|T < 7315742318451613696}]
I20260807 02:31:41.993444 19294 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.993546 19294 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.996199 19306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.996327 19306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:41.997468 19274 sys_catalog.cc:455] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a5c73942aa474a9cdb80fd75ff0af2" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 41935 } } peers { permanent_uuid: "b005fbad47064eec92765f744bbd2bfc" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 36299 } } peers { permanent_uuid: "fe92f9f9d1f14391910212af26372959" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 43601 } } }
I20260807 02:31:41.997575 19274 sys_catalog.cc:458] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:41.998019 19336 mvcc.cc:205] Tried to move back new op lower bound from 7315742318451613696 to 7315742318373507072. Current Snapshot: MvccSnapshot[applied={T|T < 7315742318451613696}]
I20260807 02:31:41.998459 19337 mvcc.cc:205] Tried to move back new op lower bound from 7315742318451613696 to 7315742318373507072. Current Snapshot: MvccSnapshot[applied={T|T < 7315742318451613696}]
I20260807 02:31:42.002218 19325 catalog_manager.cc:1347] Generated new cluster ID: c728dc30dcb54cfeb212d548f9009721
I20260807 02:31:42.002295 19325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:42.028656 19325 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:42.030045 19325 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:42.049014 19325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80a5c73942aa474a9cdb80fd75ff0af2: Generated new TSK 0
I20260807 02:31:42.050000 19325 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:31:42.086330 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.083001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.17.53.61:36299)
0807 02:31:42.083147 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.083153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.084890 (+ 1737us) server_negotiation.cc:407] Connection header received
0807 02:31:42.085377 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.085380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.085439 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.085496 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.086172 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43838: BlockingRecv error: recv got EOF from 127.0.0.1:43838 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 02:31:42.087721 19032 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.083891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.17.53.62:41935)
0807 02:31:42.084016 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.084021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.086829 (+ 2808us) server_negotiation.cc:407] Connection header received
0807 02:31:42.087004 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.087007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.087057 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.087107 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.087606 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60992: BlockingRecv error: recv got EOF from 127.0.0.1:60992 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 02:31:42.088959 19250 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.084571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.17.53.60:43601)
0807 02:31:42.084722 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.084727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.088099 (+ 3372us) server_negotiation.cc:407] Connection header received
0807 02:31:42.088202 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.088205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.088276 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.088371 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.088823 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45446: BlockingRecv error: recv got EOF from 127.0.0.1:45446 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 02:31:42.093156 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.091496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.17.53.61:36299)
0807 02:31:42.091605 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.091609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.092081 (+ 472us) server_negotiation.cc:407] Connection header received
0807 02:31:42.092314 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.092318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.092400 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.092456 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.093052 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43840: BlockingRecv error: recv got EOF from 127.0.0.1:43840 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 02:31:42.098341 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.096658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.17.53.61:36299)
0807 02:31:42.096789 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.096794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.097361 (+ 567us) server_negotiation.cc:407] Connection header received
0807 02:31:42.097554 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.097557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.097607 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.097659 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.098229 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43854: BlockingRecv error: recv got EOF from 127.0.0.1:43854 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 02:31:42.104040 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.102462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.17.53.61:36299)
0807 02:31:42.102564 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.102568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.103089 (+ 521us) server_negotiation.cc:407] Connection header received
0807 02:31:42.103302 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.103307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.103360 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.103412 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.103927 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43860: BlockingRecv error: recv got EOF from 127.0.0.1:43860 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 02:31:42.111907 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.110181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.17.53.61:36299)
0807 02:31:42.110346 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.110350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.110915 (+ 565us) server_negotiation.cc:407] Connection header received
0807 02:31:42.111127 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.111131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.111185 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.111237 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.111796 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43870: BlockingRecv error: recv got EOF from 127.0.0.1:43870 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 02:31:42.124393 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.122530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.17.53.61:36299)
0807 02:31:42.122660 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.122665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.123290 (+ 625us) server_negotiation.cc:407] Connection header received
0807 02:31:42.123529 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.123533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.123592 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.123654 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.124260 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43874: BlockingRecv error: recv got EOF from 127.0.0.1:43874 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 02:31:42.145533 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.143294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43876 (local address 127.17.53.61:36299)
0807 02:31:42.143439 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.143444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.144193 (+ 749us) server_negotiation.cc:407] Connection header received
0807 02:31:42.144478 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.144481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.144562 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.144618 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.145370 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43876: BlockingRecv error: recv got EOF from 127.0.0.1:43876 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 02:31:42.181561 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.180148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.17.53.61:36299)
0807 02:31:42.180297 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.180302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.180575 (+ 273us) server_negotiation.cc:407] Connection header received
0807 02:31:42.180790 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.180797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.180860 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.180949 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.181435 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43878: BlockingRecv error: recv got EOF from 127.0.0.1:43878 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 02:31:42.248986 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.247641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.17.53.61:36299)
0807 02:31:42.247778 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.247783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.248008 (+ 225us) server_negotiation.cc:407] Connection header received
0807 02:31:42.248190 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.248194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.248254 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.248355 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.248873 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43892: BlockingRecv error: recv got EOF from 127.0.0.1:43892 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 02:31:42.382637 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.381403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.17.53.61:36299)
0807 02:31:42.381562 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.381567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.381751 (+ 184us) server_negotiation.cc:407] Connection header received
0807 02:31:42.381895 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.381897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.381951 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.382026 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.382505 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43900: BlockingRecv error: recv got EOF from 127.0.0.1:43900 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 02:31:42.642277 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:42.640930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.17.53.61:36299)
0807 02:31:42.641114 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:42.641120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:42.641263 (+ 143us) server_negotiation.cc:407] Connection header received
0807 02:31:42.641453 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:42.641457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:42.641519 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:42.641616 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:42.642129 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43916: BlockingRecv error: recv got EOF from 127.0.0.1:43916 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 02:31:42.952685 19322 catalog_manager.cc:1259] Loaded cluster ID: c728dc30dcb54cfeb212d548f9009721
I20260807 02:31:42.952800 19322 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: loading cluster ID for follower catalog manager: success
I20260807 02:31:42.954543 19322 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: acquiring CA information for follower catalog manager: success
I20260807 02:31:42.955770 19322 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe92f9f9d1f14391910212af26372959: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:31:42.959313 19291 catalog_manager.cc:1259] Loaded cluster ID: c728dc30dcb54cfeb212d548f9009721
I20260807 02:31:42.959406 19291 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: loading cluster ID for follower catalog manager: success
I20260807 02:31:42.961023 19291 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: acquiring CA information for follower catalog manager: success
I20260807 02:31:42.962182 19291 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b005fbad47064eec92765f744bbd2bfc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:31:43.161273 19437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:43.156469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.17.53.61:36299)
0807 02:31:43.156841 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:43.156847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:43.156867 (+ 20us) server_negotiation.cc:407] Connection header received
0807 02:31:43.157067 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:43.157070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:43.157218 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:43.157351 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:43.157664 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.158583 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.159546 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.159816 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.159938 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:43.160311 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:43.160826 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:43.160980 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:43.161027 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 02:31:43.161090 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":71,"threads_started":1}
I20260807 02:31:43.168483 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:43.164712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.17.53.60:43601)
0807 02:31:43.164983 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:43.164987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:43.165017 (+ 30us) server_negotiation.cc:407] Connection header received
0807 02:31:43.165099 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:43.165101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:43.165213 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:43.165312 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:43.165532 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.166063 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.167037 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.167257 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.167339 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:43.167638 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:43.168050 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:43.168139 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:43.168238 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 02:31:43.168291 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":101,"threads_started":1}
I20260807 02:31:43.176081 19437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:43.171901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.17.53.61:36299)
0807 02:31:43.172054 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:43.172060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:43.172405 (+ 345us) server_negotiation.cc:407] Connection header received
0807 02:31:43.172590 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:43.172595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:43.172651 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:43.172739 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:43.173092 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.173922 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.174890 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.175148 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.175260 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:43.175506 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:43.175601 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:43.175678 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:43.175839 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 02:31:43.175898 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 02:31:43.176503 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:43.172288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.17.53.60:43601)
0807 02:31:43.172408 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:43.172412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:43.172600 (+ 188us) server_negotiation.cc:407] Connection header received
0807 02:31:43.172739 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:43.172742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:43.172852 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:43.172920 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:43.173244 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.173805 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.175042 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.175385 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.175466 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:43.176006 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:43.176091 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:43.176139 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:43.176324 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 02:31:43.176375 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:31:43.177052 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:43.171870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.17.53.62:41935)
0807 02:31:43.172157 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:43.172162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:43.172352 (+ 190us) server_negotiation.cc:407] Connection header received
0807 02:31:43.172544 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:43.172551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:43.172797 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:43.172933 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:43.173210 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.173993 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.175161 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:43.175491 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:43.175601 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:43.176061 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:43.176425 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:43.176506 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:43.176831 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 02:31:43.176894 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260807 02:31:43.180258 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 18919
I20260807 02:31:43.201066 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19036
I20260807 02:31:43.219834 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19140
2026-08-07T02:31:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3414 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 02:31:43 dist-test-slave-g170 krb5kdc[19459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 02:31:43 dist-test-slave-g170 krb5kdc[19459](info): set up 2 sockets
Aug 07 02:31:43 dist-test-slave-g170 krb5kdc[19459](info): commencing operation
krb5kdc: starting...
W20260807 02:31:44.472748 17620 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/test-user.keytab.
Aug 07 02:31:44 dist-test-slave-g170 krb5kdc[19459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T02:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T02:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.53.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
Entry for principal kudu/127.17.53.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab.
I20260807 02:31:44.602066 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35509
--webserver_interface=127.17.53.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 02:31:44.722662 19480 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:44.722971 19480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:44.722997 19480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:44.723035 19480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:44.727077 19480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:44.727145 19480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:44.727165 19480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:44.727182 19480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:44.727197 19480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:44.732646 19480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.62.keytab
--principal=kudu/127.17.53.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.62:35509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.53.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:44.733840 19480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:44.734876 19480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:44.744194 19487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:44.744187 19486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:44.744187 19490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:44.744335 19480 server_base.cc:1034] running on GCE node
I20260807 02:31:44.745247 19480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:44.746500 19480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:44.747715 19480 hybrid_clock.cc:648] HybridClock initialized: now 1786069904747682 us; error 46 us; skew 500 ppm
Aug 07 02:31:44 dist-test-slave-g170 krb5kdc[19459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:44.750746 19480 init.cc:377] Logged in from keytab as kudu/127.17.53.62@KRBTEST.COM (short username kudu)
I20260807 02:31:44.751750 19480 webserver.cc:460] Webserver started at http://127.17.53.62:36735/ using document root <none> and password file <none>
I20260807 02:31:44.752377 19480 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:44.752475 19480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:44.752717 19480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:44.754889 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data/instance:
uuid: "83404379db7e4558936d7d6bf9f47099"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "2c21c9a3f7921576f467343d1a39bfab"
server_key_iv: "3e088c963bcb10117d58acfce4d7f5fe"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.755477 19480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance:
uuid: "83404379db7e4558936d7d6bf9f47099"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "2c21c9a3f7921576f467343d1a39bfab"
server_key_iv: "3e088c963bcb10117d58acfce4d7f5fe"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.759490 19480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 02:31:44.762029 19498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:44.763360 19480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:31:44.763511 19480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal
uuid: "83404379db7e4558936d7d6bf9f47099"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "2c21c9a3f7921576f467343d1a39bfab"
server_key_iv: "3e088c963bcb10117d58acfce4d7f5fe"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.763623 19480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:44.787802 19480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:44.791060 19480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:44.791276 19480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:44.799219 19480 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.62:35509
I20260807 02:31:44.799540 19583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.62:35509 every 8 connection(s)
I20260807 02:31:44.800343 19480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/data/info.pb
I20260807 02:31:44.803187 19584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:44.807070 19584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:44.809449 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19480
I20260807 02:31:44.809634 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-0/wal/instance
I20260807 02:31:44.809942 17620 external_mini_cluster.cc:1428] Setting key 060be389ddb83f5cde4d1e1730139581
Aug 07 02:31:44 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.53.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.61@KRBTEST.COM" created.
I20260807 02:31:44.822813 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.809297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35509 (local address 127.0.0.1:49064)
0807 02:31:44.810033 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.810087 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:44.810940 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:44.811621 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:44.811627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:44.812058 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:44.812318 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:44.812331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:44.813536 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:44.813539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:44.814264 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:44.814273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:44.814901 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:44.814905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:44.815017 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:44.815635 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:44.815651 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:44.817513 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:44.819705 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:44.819716 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:44.819727 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:44.819996 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:44.820324 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:44.820330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:44.820332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:44.820453 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:44.821113 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:44.821120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:44.821262 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 02:31:44.821435 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 02:31:44.821832 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":138,"thread_start_us":131,"threads_started":1}
I20260807 02:31:44.822817 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.809296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49064 (local address 127.17.53.62:35509)
0807 02:31:44.809757 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:44.809777 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:44.810205 (+ 428us) server_negotiation.cc:407] Connection header received
0807 02:31:44.811251 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:44.811256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:44.811440 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:44.811622 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:44.812470 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:44.813409 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:44.814386 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:44.814751 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:44.814919 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:44.817666 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:44.817672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:44.817682 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:44.817703 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:44.819554 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:44.820105 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:44.820116 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:44.820117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:44.820223 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:44.820544 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:44.820549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:44.820551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:44.820996 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:44.821130 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:44.821499 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 02:31:44.821777 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":302,"thread_start_us":140,"threads_started":1}
I20260807 02:31:44.825583 19584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:44.826368 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.826094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44690)
0807 02:31:44.826217 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.826291 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 02:31:44.826568 19500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
I20260807 02:31:44.827396 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.827175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44702)
0807 02:31:44.827281 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.827331 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 02:31:44.827575 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:33721: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
Entry for principal kudu/127.17.53.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab.
I20260807 02:31:44.838119 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:33721
--webserver_interface=127.17.53.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:31:44.876849 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } attempt: 1
I20260807 02:31:44.877754 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.877456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44708)
0807 02:31:44.877612 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.877675 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 02:31:44.878494 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.878260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44714)
0807 02:31:44.878383 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.878434 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 02:31:44.878623 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:33721: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
I20260807 02:31:44.946776 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } attempt: 2
I20260807 02:31:44.947666 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.947372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44722)
0807 02:31:44.947523 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.947587 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 02:31:44.948400 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:44.948188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44732)
0807 02:31:44.948297 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:44.948346 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 02:31:44.948514 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.61:33721: Network error: Client connection negotiation failed: client connection to 127.17.53.61:33721: connect: Connection refused (error 111)
W20260807 02:31:44.954772 19594 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:44.955068 19594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:44.955124 19594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:44.955197 19594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:44.959314 19594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:44.959390 19594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:44.959426 19594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:44.959477 19594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:44.959513 19594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:44.964959 19594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.61.keytab
--principal=kudu/127.17.53.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.61:33721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.53.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:44.966127 19594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:44.967236 19594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:44.975872 19606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:44.975878 19605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:44.975956 19610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:44.976316 19594 server_base.cc:1034] running on GCE node
I20260807 02:31:44.976994 19594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:44.978116 19594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:44.979315 19594 hybrid_clock.cc:648] HybridClock initialized: now 1786069904979283 us; error 49 us; skew 500 ppm
Aug 07 02:31:44 dist-test-slave-g170 krb5kdc[19459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:44.982240 19594 init.cc:377] Logged in from keytab as kudu/127.17.53.61@KRBTEST.COM (short username kudu)
I20260807 02:31:44.983150 19594 webserver.cc:460] Webserver started at http://127.17.53.61:46369/ using document root <none> and password file <none>
I20260807 02:31:44.983729 19594 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:44.983825 19594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:44.984067 19594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:44.986260 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data/instance:
uuid: "04fa03834ea54717a4dd0de6dee882f5"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "98fb97c0459ba369c9a81b31e3fe3bb8"
server_key_iv: "ff7832d94f935b5a2f71d5a1c12e5907"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.986864 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal/instance:
uuid: "04fa03834ea54717a4dd0de6dee882f5"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "98fb97c0459ba369c9a81b31e3fe3bb8"
server_key_iv: "ff7832d94f935b5a2f71d5a1c12e5907"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.990811 19594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 02:31:44.993278 19620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:44.994550 19594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 02:31:44.994688 19594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal
uuid: "04fa03834ea54717a4dd0de6dee882f5"
format_stamp: "Formatted at 2026-08-07 02:31:44 on dist-test-slave-g170"
server_key: "98fb97c0459ba369c9a81b31e3fe3bb8"
server_key_iv: "ff7832d94f935b5a2f71d5a1c12e5907"
server_key_version: "encryptionkey@0"
I20260807 02:31:44.994793 19594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:45.012533 19594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:45.016072 19594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:45.016396 19594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:45.023963 19594 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.61:33721
I20260807 02:31:45.024276 19698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.61:33721 every 8 connection(s)
I20260807 02:31:45.025058 19594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/data/info.pb
I20260807 02:31:45.025426 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19594
I20260807 02:31:45.025599 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-1/wal/instance
I20260807 02:31:45.025965 17620 external_mini_cluster.cc:1428] Setting key b2d1bdea6fb18943e382311bc9d41192
I20260807 02:31:45.028743 19699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:45.032639 19699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.53.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.53.60@KRBTEST.COM" created.
I20260807 02:31:45.039674 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } attempt: 3
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:31:45.048216 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.034582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.17.53.62:35509)
0807 02:31:45.034745 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.034752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.035539 (+ 787us) server_negotiation.cc:407] Connection header received
0807 02:31:45.036444 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.036447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.036499 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.036580 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.037909 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.038813 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.040884 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.041325 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.041451 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.044124 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.044132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.044135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.044156 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.045899 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.046407 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.046413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.046415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.046509 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.046942 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.046947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.046950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.047164 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.047281 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.047897 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.048044 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 02:31:45.049253 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.034913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35509 (local address 127.0.0.1:49078)
0807 02:31:45.035374 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.035417 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.036242 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.036826 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.036833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.037186 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.037755 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.037769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.038941 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.038943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.040719 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.040731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.041515 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.041519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.041706 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.042397 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.042417 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.043984 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.046011 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.046017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.046031 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.046291 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.046627 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.046631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.046633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.046823 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.047296 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.047303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.047619 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.047838 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.048144 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260807 02:31:45.050877 19699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:45.051787 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.040116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44748)
0807 02:31:45.040300 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.040312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.040372 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.041006 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.041009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.041205 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.041314 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.041320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.042553 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.042560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.043515 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.043530 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.044469 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.044474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.044601 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.045699 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.045717 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.047187 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.050011 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.050013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.050015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.050256 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.050581 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.050583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.050584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.050659 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.051196 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.051199 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.051316 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.051430 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.051576 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2}
I20260807 02:31:45.052402 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.040215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.17.53.61:33721)
0807 02:31:45.040592 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.040600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.040620 (+ 20us) server_negotiation.cc:407] Connection header received
0807 02:31:45.040653 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.040657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.040859 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.041016 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.041426 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.042425 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.043680 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.044318 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.044448 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.047343 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.047350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.047362 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.047386 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.049912 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.050393 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.050400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.050402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.050493 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.050778 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.050781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.050783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.051092 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.051229 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.052136 (+ 907us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.052272 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":190,"threads_started":1}
I20260807 02:31:45.054062 19584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
Entry for principal kudu/127.17.53.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab.
I20260807 02:31:45.054826 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.054545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45400)
0807 02:31:45.054687 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.054749 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 02:31:45.055617 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.055320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45408)
0807 02:31:45.055488 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.055540 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260807 02:31:45.055763 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:31:45.055868 17620 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
/tmp/dist-test-taskPRMQP4/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:46339
--webserver_interface=127.17.53.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.53.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 02:31:45.060586 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.051410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.17.53.61:33721)
0807 02:31:45.051709 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.051716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.051728 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:31:45.051753 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.051755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.051785 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.051876 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.052265 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.052954 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.053770 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.054030 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.054127 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.056493 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.056499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.056505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.056544 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.058851 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.059302 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.059304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.059306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.059383 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.059680 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.059682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.059683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.059885 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.059962 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.060280 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.060443 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":238,"thread_start_us":103,"threads_started":1}
I20260807 02:31:45.060680 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.051353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44758)
0807 02:31:45.051440 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.051453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.051525 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.051885 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.051888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.052045 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.052163 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.052173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.053086 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.053092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.053624 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.053633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.054171 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.054175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.054320 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.054839 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.054858 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.056337 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.058964 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.058969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.058973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.059213 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.059467 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.059473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.059478 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.059583 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.059981 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.059987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.060112 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.060232 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.060390 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 02:31:45.061723 19699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:45.062426 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.062180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45422)
0807 02:31:45.062297 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.062373 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 02:31:45.062726 19623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.063637 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.063431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45424)
0807 02:31:45.063535 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.063583 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 02:31:45.063880 19699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.086920 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 1
I20260807 02:31:45.087751 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.087466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45438)
0807 02:31:45.087615 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.087677 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 02:31:45.088449 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.088239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45454)
0807 02:31:45.088344 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.088397 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 02:31:45.088599 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.113112 19699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 1
I20260807 02:31:45.114019 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.113719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45470)
0807 02:31:45.113873 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.113937 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 02:31:45.114811 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.114556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45486)
0807 02:31:45.114677 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.114754 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 02:31:45.114950 19699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.163794 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 2
I20260807 02:31:45.164853 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.164431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45494)
0807 02:31:45.164627 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.164710 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 02:31:45.165838 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.165542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45498)
0807 02:31:45.165693 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.165758 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 02:31:45.166035 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.183105 19699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 2
I20260807 02:31:45.184109 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.183700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45504)
0807 02:31:45.183925 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.184001 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 02:31:45.185145 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.184836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45510)
0807 02:31:45.185004 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.185064 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260807 02:31:45.185410 19699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
W20260807 02:31:45.188692 19706 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 02:31:45.189073 19706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 02:31:45.189117 19706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 02:31:45.189182 19706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 02:31:45.193250 19706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 02:31:45.193305 19706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 02:31:45.193323 19706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 02:31:45.193359 19706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 02:31:45.193392 19706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 02:31:45.198782 19706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.53.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.53.62:35509,127.17.53.61:33721,127.17.53.60:46339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/krb5kdc/kudu_127.17.53.60.keytab
--principal=kudu/127.17.53.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.53.60:46339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.53.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 01:47:39 UTC on ccycloud.quasar-ogxevj.root.comops.site
build id 4571737
I20260807 02:31:45.199889 19706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 02:31:45.200975 19706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 02:31:45.209507 19719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:45.209523 19714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 02:31:45.209518 19713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 02:31:45.209589 19706 server_base.cc:1034] running on GCE node
I20260807 02:31:45.210295 19706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 02:31:45.211366 19706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 02:31:45.212558 19706 hybrid_clock.cc:648] HybridClock initialized: now 1786069905212532 us; error 37 us; skew 500 ppm
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 02:31:45.215601 19706 init.cc:377] Logged in from keytab as kudu/127.17.53.60@KRBTEST.COM (short username kudu)
I20260807 02:31:45.216511 19706 webserver.cc:460] Webserver started at http://127.17.53.60:39499/ using document root <none> and password file <none>
I20260807 02:31:45.217144 19706 fs_manager.cc:359] Metadata directory not provided
I20260807 02:31:45.217233 19706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 02:31:45.217464 19706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 02:31:45.219482 19706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data/instance:
uuid: "0a9db171ab8e46afacedc1871deff8ea"
format_stamp: "Formatted at 2026-08-07 02:31:45 on dist-test-slave-g170"
server_key: "0c139ae16cb1ba38492feda43dfad986"
server_key_iv: "70378e36324fd73d4db9d8e43b1f610d"
server_key_version: "encryptionkey@0"
I20260807 02:31:45.220039 19706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal/instance:
uuid: "0a9db171ab8e46afacedc1871deff8ea"
format_stamp: "Formatted at 2026-08-07 02:31:45 on dist-test-slave-g170"
server_key: "0c139ae16cb1ba38492feda43dfad986"
server_key_iv: "70378e36324fd73d4db9d8e43b1f610d"
server_key_version: "encryptionkey@0"
I20260807 02:31:45.223891 19706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 02:31:45.226437 19731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 02:31:45.227720 19706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 02:31:45.227859 19706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data,/tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal
uuid: "0a9db171ab8e46afacedc1871deff8ea"
format_stamp: "Formatted at 2026-08-07 02:31:45 on dist-test-slave-g170"
server_key: "0c139ae16cb1ba38492feda43dfad986"
server_key_iv: "70378e36324fd73d4db9d8e43b1f610d"
server_key_version: "encryptionkey@0"
I20260807 02:31:45.227968 19706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 02:31:45.258002 19706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 02:31:45.261076 19706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 02:31:45.261284 19706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 02:31:45.265208 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 3
I20260807 02:31:45.265949 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.265659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45512)
0807 02:31:45.265794 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.265861 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 02:31:45.266794 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.266534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45528)
0807 02:31:45.266640 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.266712 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 02:31:45.267002 19584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.53.60:46339: Network error: Client connection negotiation failed: client connection to 127.17.53.60:46339: connect: Connection refused (error 111)
I20260807 02:31:45.269400 19706 rpc_server.cc:307] RPC server started. Bound to: 127.17.53.60:46339
I20260807 02:31:45.269680 19811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.53.60:46339 every 8 connection(s)
I20260807 02:31:45.270414 19706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/data/info.pb
I20260807 02:31:45.273567 19812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 02:31:45.274176 17620 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu as pid 19706
I20260807 02:31:45.274307 17620 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPRMQP4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786069602236550-17620-0/minicluster-data/master-2/wal/instance
I20260807 02:31:45.274662 17620 external_mini_cluster.cc:1428] Setting key 2639b0cb469b90126305c78e17d0f3ac
I20260807 02:31:45.276583 19699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 3
I20260807 02:31:45.279400 19812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.61@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:45.290103 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.277095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45540)
0807 02:31:45.277268 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.277285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.277395 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.279447 (+ 2052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.279450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.279664 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.279789 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.279797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.280907 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.280912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.281608 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.281618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.282676 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.282680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.282772 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.283356 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.283369 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.284832 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.287437 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.287442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.287445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.287750 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.288176 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.288179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.288182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.288283 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.289213 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.289217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.289568 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.289715 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.289883 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 02:31:45.291342 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.277193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.17.53.60:46339)
0807 02:31:45.277507 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.277517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.277534 (+ 17us) server_negotiation.cc:407] Connection header received
0807 02:31:45.277569 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.277571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.278034 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.278419 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.279981 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.280741 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.281738 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.282535 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.282721 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.285032 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.285043 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.285070 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.285113 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.287297 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.287888 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.287897 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.287905 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.288055 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.288432 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.288439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.288444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.289078 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.289257 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.289893 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.290204 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":243,"spinlock_wait_cycles":5760,"thread_start_us":138,"threads_started":1}
I20260807 02:31:45.291575 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.276003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.17.53.60:46339)
0807 02:31:45.276497 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.276513 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.276548 (+ 35us) server_negotiation.cc:407] Connection header received
0807 02:31:45.277541 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.277566 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.278034 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.278419 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.279390 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.280740 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.282922 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.283710 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.283822 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.286725 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.286729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.286732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.286760 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.289467 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.290000 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.290004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.290006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.290117 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.290522 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.290527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.290530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.290778 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.290978 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.291215 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.291346 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":351,"spinlock_wait_cycles":25216,"thread_start_us":178,"threads_started":1}
I20260807 02:31:45.292217 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.281360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.17.53.62:35509)
0807 02:31:45.281507 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.281512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.281897 (+ 385us) server_negotiation.cc:407] Connection header received
0807 02:31:45.282112 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.282115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.282166 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.282249 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.283020 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.283893 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.285118 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.285519 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.285639 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.288473 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.288476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.288479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.288511 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.290109 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.290639 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.290642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.290644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.290731 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.291041 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.291045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.291047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.291259 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.291345 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.291946 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.292075 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 02:31:45.292801 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.281291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.62:35509 (local address 127.0.0.1:49094)
0807 02:31:45.281740 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.281820 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.282002 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.282302 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.282308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.282705 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.282873 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.282883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.284024 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.284031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.284989 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.284999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.285992 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.285997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.286192 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.286699 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.286712 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.288336 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.290234 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.290242 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.290247 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.290516 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.290846 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.290849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.290851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.290937 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.291581 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.291587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.291708 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.292419 (+ 711us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.292596 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":111,"threads_started":1}
I20260807 02:31:45.293979 19812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } has no permanent_uuid. Determining permanent_uuid...
I20260807 02:31:45.296494 19699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: Bootstrap starting.
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:31:45.299582 19699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:45.300541 19699 log.cc:824] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:45.302953 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.294552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.17.53.61:33721)
0807 02:31:45.294691 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.294695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.294728 (+ 33us) server_negotiation.cc:407] Connection header received
0807 02:31:45.294840 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.294845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.294889 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.294965 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.295341 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.296092 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.297022 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.297343 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.297454 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.299505 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.299508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.299510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.299531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.300987 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.301486 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.301491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.301494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.301587 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.301952 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.301957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.301959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.302241 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.302360 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.302680 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.302809 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 02:31:45.303081 19699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: No bootstrap required, opened a new log
I20260807 02:31:45.303417 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.294435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.61:33721 (local address 127.0.0.1:44774)
0807 02:31:45.294674 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.294686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.294758 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.294999 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.295001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.295126 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.295215 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.295219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.296195 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.296197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.296915 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.296924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.297449 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.297452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.297520 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.297983 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.297994 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.299377 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.301106 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.301113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.301118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.301364 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.301726 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.301729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.301731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.301826 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.302370 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.302373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.302469 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.303136 (+ 667us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.303270 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260807 02:31:45.304075 19812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.62@KRBTEST.COM
I20260807 02:31:45.307802 19699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.308122 19699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fa03834ea54717a4dd0de6dee882f5, State: Initialized, Role: FOLLOWER
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.60@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:31:45.308626 19699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.310349 19699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:45.310563 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.301325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.17.53.62:35509)
0807 02:31:45.301449 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.301453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.301540 (+ 87us) server_negotiation.cc:407] Connection header received
0807 02:31:45.301672 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.301674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.301710 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.301798 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.302524 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.303017 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.303969 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.304260 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.304366 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.307178 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.307183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.307186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.307215 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.308841 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.309340 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.309345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.309347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.309430 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.309723 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.309726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.309728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.309911 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.310146 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.310277 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.310390 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:31:45.311221 19824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.311478 19824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.313436 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.304583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45552)
0807 02:31:45.304733 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.304745 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.304889 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.305109 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.305112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.305268 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.305325 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.305330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.306014 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.306020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.306503 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.306510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.307002 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.307005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.307084 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.307505 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.307516 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.308958 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.311362 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.311369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.311374 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.311668 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.312066 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.312068 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.312071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.312166 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.312754 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.312793 (+ 39us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.312905 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.313066 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.313222 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 02:31:45.313544 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.304591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.17.53.60:46339)
0807 02:31:45.304689 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.304695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.304856 (+ 161us) server_negotiation.cc:407] Connection header received
0807 02:31:45.304982 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.304985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.305019 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.305098 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.305427 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.305908 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.306618 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.306891 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.306990 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.309090 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.309094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.309096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.309130 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.310911 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.311846 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.311850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.311852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.311948 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.312278 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.312281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.312283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.312597 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.312678 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.313156 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.313293 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.53.61@KRBTEST.COM
I20260807 02:31:45.318504 19812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: Bootstrap starting.
I20260807 02:31:45.315413 19594 master_runner.cc:186] Error getting master registration for 127.17.53.62:35509: OK, instance_id { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" instance_seqno: 1786069904797132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:31:45.318955 19594 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 02:31:45.319290 19841 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:45.319411 19841 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:45.321906 19812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:45.322978 19812 log.cc:824] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:45.323854 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.312162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.17.53.61:33721)
0807 02:31:45.312334 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.312341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.312401 (+ 60us) server_negotiation.cc:407] Connection header received
0807 02:31:45.312549 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.312553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.312601 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.312698 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.313580 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.314343 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.315506 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.315839 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.315955 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.319258 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.319262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.319265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.319293 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.321236 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.321796 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.321802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.321807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.321912 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.322293 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.322298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.322300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.322546 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.322788 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.323516 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.323651 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 02:31:45.325240 19812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: No bootstrap required, opened a new log
I20260807 02:31:45.329175 19812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.329487 19812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9db171ab8e46afacedc1871deff8ea, State: Initialized, Role: FOLLOWER
I20260807 02:31:45.329874 19812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.330466 19845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.330685 19845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.331019 17620 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 02:31:45.331490 19812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:45.332446 19540 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "83404379db7e4558936d7d6bf9f47099"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49108
W20260807 02:31:45.337844 19860 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:45.337970 19860 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:45.335592 19706 master_runner.cc:186] Error getting master registration for 127.17.53.62:35509: OK, instance_id { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" instance_seqno: 1786069904797132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 02:31:45.339119 19706 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:31:45.431185 19584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } attempt: 4
Aug 07 02:31:45 dist-test-slave-g170 krb5kdc[19459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786069904, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.53.62@KRBTEST.COM for kudu/127.17.53.60@KRBTEST.COM
I20260807 02:31:45.442778 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.431819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.53.60:46339 (local address 127.0.0.1:45558)
0807 02:31:45.432016 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 02:31:45.432035 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 02:31:45.432161 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 02:31:45.433111 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 02:31:45.433115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 02:31:45.433358 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 02:31:45.433482 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.433489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.434741 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.434745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.435598 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 02:31:45.435610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.436254 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.436258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 02:31:45.436354 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.436928 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 02:31:45.436945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 02:31:45.438654 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 02:31:45.440945 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.440948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.440951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.441231 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.441565 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 02:31:45.441569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 02:31:45.441571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 02:31:45.441672 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 02:31:45.442183 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 02:31:45.442186 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 02:31:45.442294 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 02:31:45.442403 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 02:31:45.442569 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 02:31:45.443418 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.432412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.17.53.60:46339)
0807 02:31:45.432541 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.432546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.432560 (+ 14us) server_negotiation.cc:407] Connection header received
0807 02:31:45.432608 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.432612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.432693 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.432913 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.433638 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.434603 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.435727 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.436118 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.436232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.438843 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.438848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.438851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.438882 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.440796 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.441358 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.441362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.441365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.441457 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.441795 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.441799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.441801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.442037 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.442135 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.443105 (+ 970us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.443248 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 02:31:45.447357 19584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: Bootstrap starting.
I20260807 02:31:45.450700 19584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: Neither blocks nor log segments found. Creating new log.
I20260807 02:31:45.451735 19584 log.cc:824] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: Log is configured to *not* fsync() on all Append() calls
I20260807 02:31:45.454401 19584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: No bootstrap required, opened a new log
I20260807 02:31:45.458395 19584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.458737 19584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83404379db7e4558936d7d6bf9f47099, State: Initialized, Role: FOLLOWER
I20260807 02:31:45.459177 19584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.459924 19865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.460126 19865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.461045 19584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 02:31:45.461117 19540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 02:31:45.461247 19540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:45.462554 19540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.465049 19778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83404379db7e4558936d7d6bf9f47099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0a9db171ab8e46afacedc1871deff8ea"
I20260807 02:31:45.465317 19778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:45.466753 19778 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83404379db7e4558936d7d6bf9f47099 in term 1.
I20260807 02:31:45.467689 19501 leader_election.cc:302] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a9db171ab8e46afacedc1871deff8ea, 83404379db7e4558936d7d6bf9f47099; no voters:
W20260807 02:31:45.468415 19881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 02:31:45.468569 19881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 02:31:45.468887 19662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83404379db7e4558936d7d6bf9f47099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04fa03834ea54717a4dd0de6dee882f5"
I20260807 02:31:45.469197 19662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 02:31:45.470486 19662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83404379db7e4558936d7d6bf9f47099 in term 1.
I20260807 02:31:45.463632 19540 leader_election.cc:288] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [CANDIDATE]: Term 1 election: Requested vote from peers 04fa03834ea54717a4dd0de6dee882f5 (127.17.53.61:33721), 0a9db171ab8e46afacedc1871deff8ea (127.17.53.60:46339)
I20260807 02:31:45.477581 19480 master_runner.cc:263] No leader master found from master 83404379db7e4558936d7d6bf9f47099
I20260807 02:31:45.477654 19480 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 02:31:45.477744 19865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 02:31:45.477953 19865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [term 1 LEADER]: Becoming Leader. State: Replica: 83404379db7e4558936d7d6bf9f47099, State: Running, Role: LEADER
I20260807 02:31:45.478307 19865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } }
I20260807 02:31:45.480082 19865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83404379db7e4558936d7d6bf9f47099. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.480201 19865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:45.491118 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.480791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.17.53.60:46339)
0807 02:31:45.480941 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.480947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.481600 (+ 653us) server_negotiation.cc:407] Connection header received
0807 02:31:45.481809 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.481814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.481866 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.481920 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.482990 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.483788 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.485228 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.485546 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.485627 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.487435 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.487439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.487441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.487467 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.489128 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.489643 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.489647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.489649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.489746 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.490072 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.490076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.490078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.490318 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.490427 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.490806 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.490932 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 02:31:45.492301 19891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 02:31:45.493443 19891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 02:31:45.502759 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.481674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.17.53.61:33721)
0807 02:31:45.481805 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.481809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.493020 (+ 11211us) server_negotiation.cc:407] Connection header received
0807 02:31:45.493218 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.493222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.493274 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.493328 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.494239 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.494958 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.496011 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.496358 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.496427 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.498585 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.498589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.498593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.498686 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.500655 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.501251 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.501255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.501257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.501358 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.501700 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.501703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.501705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.501948 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.502067 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.502452 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.502580 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:31:45.504406 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:45.481056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.17.53.62:35509)
0807 02:31:45.481171 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.481177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.489123 (+ 7946us) server_negotiation.cc:407] Connection header received
0807 02:31:45.491054 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.491057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.491109 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.491163 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.495743 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.496469 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.497760 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:45.498151 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:45.498245 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:45.500187 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 02:31:45.500191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 02:31:45.500194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 02:31:45.500223 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 02:31:45.502048 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.502508 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.502512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.502513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.502598 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 02:31:45.503478 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 02:31:45.503482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 02:31:45.503484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 02:31:45.503709 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 02:31:45.503819 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:45.504164 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 02:31:45.504285 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 02:31:45.509598 19778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [term 1 FOLLOWER]: Refusing update from remote peer 83404379db7e4558936d7d6bf9f47099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:45.509598 19662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [term 1 FOLLOWER]: Refusing update from remote peer 83404379db7e4558936d7d6bf9f47099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 02:31:45.510311 19889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 02:31:45.510500 19865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 02:31:45.512629 19737 tablet.cc:2369] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:45.514770 19845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83404379db7e4558936d7d6bf9f47099. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.514901 19845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: This master's current role is: FOLLOWER
W20260807 02:31:45.518047 19627 tablet.cc:2369] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 02:31:45.518110 19824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83404379db7e4558936d7d6bf9f47099. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.518253 19824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.523144 19865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.523279 19865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:45.523384 19865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.523455 19865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099 [sys.catalog]: This master's current role is: LEADER
I20260807 02:31:45.524479 19824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.524595 19824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.524716 19891 catalog_manager.cc:1347] Generated new cluster ID: 902b53c6a8674710bd1e695aafa77392
I20260807 02:31:45.524811 19891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 02:31:45.525686 19845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83404379db7e4558936d7d6bf9f47099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83404379db7e4558936d7d6bf9f47099" member_type: VOTER last_known_addr { host: "127.17.53.62" port: 35509 } } peers { permanent_uuid: "04fa03834ea54717a4dd0de6dee882f5" member_type: VOTER last_known_addr { host: "127.17.53.61" port: 33721 } } peers { permanent_uuid: "0a9db171ab8e46afacedc1871deff8ea" member_type: VOTER last_known_addr { host: "127.17.53.60" port: 46339 } } }
I20260807 02:31:45.525796 19845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea [sys.catalog]: This master's current role is: FOLLOWER
I20260807 02:31:45.526369 19901 mvcc.cc:205] Tried to move back new op lower bound from 7315742332958937088 to 7315742332842582016. Current Snapshot: MvccSnapshot[applied={T|T < 7315742332958937088}]
I20260807 02:31:45.528883 19896 mvcc.cc:205] Tried to move back new op lower bound from 7315742332958937088 to 7315742332842582016. Current Snapshot: MvccSnapshot[applied={T|T < 7315742332958937088}]
I20260807 02:31:45.538203 19891 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 02:31:45.539083 19891 catalog_manager.cc:1504] Loading token signing keys...
I20260807 02:31:45.556169 19891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83404379db7e4558936d7d6bf9f47099: Generated new TSK 0
I20260807 02:31:45.556926 19891 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 02:31:45.586975 19587 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.585318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.17.53.62:35509)
0807 02:31:45.585451 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.585456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.586058 (+ 602us) server_negotiation.cc:407] Connection header received
0807 02:31:45.586226 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.586229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.586276 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.586326 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.586844 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49134: BlockingRecv error: recv got EOF from 127.0.0.1:49134 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 02:31:45.587167 19817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.585462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.17.53.60:46339)
0807 02:31:45.585592 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.585595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.585782 (+ 187us) server_negotiation.cc:407] Connection header received
0807 02:31:45.585883 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.585888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.585958 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.586047 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.587020 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45572: BlockingRecv error: recv got EOF from 127.0.0.1:45572 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 02:31:45.588014 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.586400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.17.53.61:33721)
0807 02:31:45.586540 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.586545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.586781 (+ 236us) server_negotiation.cc:407] Connection header received
0807 02:31:45.586988 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.586994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.587047 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.587103 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.587878 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44808: BlockingRecv error: recv got EOF from 127.0.0.1:44808 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 02:31:45.591621 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.590440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.17.53.61:33721)
0807 02:31:45.590560 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.590565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.590787 (+ 222us) server_negotiation.cc:407] Connection header received
0807 02:31:45.590914 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.590918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.591000 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.591096 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.591517 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44824: BlockingRecv error: recv got EOF from 127.0.0.1:44824 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 02:31:45.595682 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.594836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.17.53.61:33721)
0807 02:31:45.594956 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.594961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.594974 (+ 13us) server_negotiation.cc:407] Connection header received
0807 02:31:45.595023 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.595027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.595086 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.595175 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.595587 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44836: BlockingRecv error: recv got EOF from 127.0.0.1:44836 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 02:31:45.601004 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.599881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.17.53.61:33721)
0807 02:31:45.600008 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.600013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.600134 (+ 121us) server_negotiation.cc:407] Connection header received
0807 02:31:45.600250 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.600253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.600297 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.600361 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.600902 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44852: BlockingRecv error: recv got EOF from 127.0.0.1:44852 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 02:31:45.608049 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.606839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.17.53.61:33721)
0807 02:31:45.606983 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.606987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.607258 (+ 271us) server_negotiation.cc:407] Connection header received
0807 02:31:45.607348 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.607351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.607404 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.607507 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.607946 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44854: BlockingRecv error: recv got EOF from 127.0.0.1:44854 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 02:31:45.618993 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.617886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.17.53.61:33721)
0807 02:31:45.618026 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.618031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.618193 (+ 162us) server_negotiation.cc:407] Connection header received
0807 02:31:45.618311 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.618314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.618388 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.618496 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.618887 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44856: BlockingRecv error: recv got EOF from 127.0.0.1:44856 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 02:31:45.638377 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.637132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.17.53.61:33721)
0807 02:31:45.637255 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.637259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.637507 (+ 248us) server_negotiation.cc:407] Connection header received
0807 02:31:45.637641 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.637643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.637692 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.637771 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.638249 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44872: BlockingRecv error: recv got EOF from 127.0.0.1:44872 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 02:31:45.674433 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.673023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.17.53.61:33721)
0807 02:31:45.673230 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.673238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.673358 (+ 120us) server_negotiation.cc:407] Connection header received
0807 02:31:45.673562 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.673565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.673619 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.673701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.674289 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44882: BlockingRecv error: recv got EOF from 127.0.0.1:44882 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 02:31:45.742283 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.740727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.17.53.61:33721)
0807 02:31:45.740991 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.740997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.741158 (+ 161us) server_negotiation.cc:407] Connection header received
0807 02:31:45.741361 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.741364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.741424 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.741526 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.742115 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44884: BlockingRecv error: recv got EOF from 127.0.0.1:44884 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
W20260807 02:31:45.873807 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:45.872501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.17.53.61:33721)
0807 02:31:45.872676 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:45.872682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:45.872846 (+ 164us) server_negotiation.cc:407] Connection header received
0807 02:31:45.873021 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:45.873024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:45.873080 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:45.873172 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:45.873664 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44888: BlockingRecv error: recv got EOF from 127.0.0.1:44888 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 02:31:46.133675 19705 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 02:31:46.132516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.17.53.61:33721)
0807 02:31:46.132662 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.132668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.132695 (+ 27us) server_negotiation.cc:407] Connection header received
0807 02:31:46.132887 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.132890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.132946 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.133045 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 02:31:46.133480 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44898: BlockingRecv error: recv got EOF from 127.0.0.1:44898 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 02:31:46.320907 19841 catalog_manager.cc:1259] Loaded cluster ID: 902b53c6a8674710bd1e695aafa77392
I20260807 02:31:46.320993 19841 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: loading cluster ID for follower catalog manager: success
I20260807 02:31:46.322896 19841 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: acquiring CA information for follower catalog manager: success
I20260807 02:31:46.324198 19841 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04fa03834ea54717a4dd0de6dee882f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:31:46.339273 19860 catalog_manager.cc:1259] Loaded cluster ID: 902b53c6a8674710bd1e695aafa77392
I20260807 02:31:46.339356 19860 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: loading cluster ID for follower catalog manager: success
I20260807 02:31:46.341322 19860 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: acquiring CA information for follower catalog manager: success
I20260807 02:31:46.342623 19860 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a9db171ab8e46afacedc1871deff8ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 02:31:46.651854 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:46.647903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.17.53.61:33721)
0807 02:31:46.648169 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.648174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.648313 (+ 139us) server_negotiation.cc:407] Connection header received
0807 02:31:46.648443 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.648446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.648577 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.648681 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:46.648973 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.649499 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.650376 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.650601 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.650683 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:46.651012 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:46.651464 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:46.651557 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:46.651631 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 02:31:46.651678 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
I20260807 02:31:46.658715 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:46.654981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.17.53.60:46339)
0807 02:31:46.655260 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.655264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.655279 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:31:46.655380 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.655382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.655493 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.655589 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:46.655853 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.656362 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.657263 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.657484 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.657561 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:46.657847 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:46.658272 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:46.658374 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:46.658463 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 02:31:46.658513 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260807 02:31:46.666244 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:46.662437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33334 (local address 127.17.53.61:33721)
0807 02:31:46.662559 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.662563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.662578 (+ 15us) server_negotiation.cc:407] Connection header received
0807 02:31:46.662700 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.662705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.662766 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.662848 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:46.663181 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.663990 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.665061 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.665320 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.665419 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:46.665775 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:46.665845 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:46.665920 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:46.666033 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 02:31:46.666084 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 02:31:46.666337 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:46.662061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.17.53.60:46339)
0807 02:31:46.662171 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.662175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.662507 (+ 332us) server_negotiation.cc:407] Connection header received
0807 02:31:46.662776 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.662781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.662837 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.662910 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:46.663284 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.664058 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.664965 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.665306 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.665381 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:46.665827 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:46.665899 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:46.665944 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:46.666127 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 02:31:46.666185 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 02:31:46.666620 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 02:31:46.661840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32874 (local address 127.17.53.62:35509)
0807 02:31:46.662276 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 02:31:46.662279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 02:31:46.662291 (+ 12us) server_negotiation.cc:407] Connection header received
0807 02:31:46.662398 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 02:31:46.662401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 02:31:46.662609 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 02:31:46.662796 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 02:31:46.663050 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.663708 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.664860 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 02:31:46.665211 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 02:31:46.665316 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 02:31:46.665642 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 02:31:46.666003 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 02:31:46.666364 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 02:31:46.666405 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 02:31:46.666476 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":74,"threads_started":1}
I20260807 02:31:46.676141 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19480
I20260807 02:31:46.690536 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19594
I20260807 02:31:46.699687 17620 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPRMQP4/build/debug/bin/kudu with pid 19706
2026-08-07T02:31:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3480 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6895 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (304456 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST